Dev/4.2.0 (#340)

* [Android]fix bytedance beauty mirror bug

* update Scene Beuaty

* update Byte Beauty

* update fu Beauty

* Update README.zh.md

* Update README.md

* [Android]fix faceunity beauty bug(CSSA-1070)

* [Android]remove inaccurate comment.

* [Android]perfect sense time beauty implement.

* [Android]adapte to 4.2.0.

* Adaptive PCM acquisition

* Add Record Moudle

* add voice formant adjust

* add video background effect

* [Android]add MediaRecorder case

* Adaptive multi-channel camera

* modify module name

* [Android]perfect MediaRecorder case

* choose screen share frameRate

* add videoProcess remote alpha mask

* [Android]add LocalVideoTranscoding case.

* [Android]add more voice configs for VoiceEffects

* [Android]add virtual background video source type for VideoProcessExtension.

* [Android]add ScreenScenarioType selector for ScreenSharing.

* [Android]update rtc version to 4.2.0 in config files

* [Android][Audio]adapt to 4.2.0 sdk.

* replace video source

* Fit the 4.2.0 Mac SDK

* add local and remote record module

* fix  record stop bug

* add virtual video background

* Mac Add local composite graph moudle

* add screen share fps choose

* open remote video alpha mask

* Fit the 4.2.0 audio-only SDK

* set voice formant

* modify mediaplayer url

* update mac media player url

* [Windows]adapt to 420 sdk and add media recoder case.

* [Windows]fix bugs of media recorder case.

* update scene resouces

* [Windows]perfect local video transcoding case.

* [Windows]add voice conversion config to beauty audio case.

* [Windows]add virtual background type choices to beauty case.

* [Windows]add screen capture scenario config.

* add video resource

* update video resource

* [Android]update test video url and file reader.

* [Android]perfect local video transcoding case

* add smaplebufferredner init method

* add multi video track moudle

* fix scene beauty change frameRate bug

* update role bug

* fix rawVideoData snapshot bug

* update save video path

* update rtmp error bug

* [Android]fix sceneTimeBeauty crash and default video resolution size problem.

* fix voice changer bug

* [Android]perfect third part beauty cases

* fix pcm play bug

* [Android]fix NMS-11565

* update script

* update script

* update exportplist file

* update script

* update scene bug

* update exportPlist file

* update exportplist file

* update script

* update script

* [Android]perfect third part beauty cases

* [Android]fix setting crash

* update script

* add logpath

* update script

* update script

* update script

* update script

* update script

* pip remote video  muted replace image

* update audio example bundle id

* update script

* update mac exportplist file

* fix mac raw video data snapshot bug

* update mac script

* update mac script

* update mac script

* update mac script

* update mac script

* update mac script

* update mac script

* update mac script

* update mac script

* update mac script

* update mac script

* update mac script

* [Windows]add PushExternalVideoYUV case.

* [Windows]fix PushExternalVideoYUV bugs.

* [Windows]add MultiVideoSourceTracks case.

* update script

* [Windows]perfect MultiVideoSourceTrack case and fix some bugs.

* [Windows]fix bug - NMS-11562

* [Windows]fix yuv raw data bug.

* [Android]add push video buffer type selector for PushExternalVideoYUV case.

* [Android]update gradle config.

* [Android]fix beauty bug in arm64.

* [Android]fix render bug in arm64

* [Android]remove ndk filter limit.

* [Windows]fix crash of switching between LiveBroadcast and AgoraScreenCapture; fix local render bug of resuming PushExternalVideoYUV.

* [Windows]perfect screen sharing case(NMS-11314) and fix some bugs.

* [Windows]fix sample res bug.

* update mac  readme

* update screen share exclude window params

* update README

* update README.

* [Android]update README.

* fix pip Android publish stream rotation 90°

* fix ci remove sources bug

* update sign mode

* update auto sign

* update script

* update script

* [Android]update byte dance camera facing config.

* fix pip display bug

* update beauty params

* Add pip version tips

* [Android]perfect MultiVideoSourceTracks case and fix some bugs.

* [Windows]fix statistic display bug and etc.

* update recorder api

* update recorder api

* pip support BGRA encode

* [Android]adjust MediaRecorder case and etc.

* [Android]fix mirror bug of scene time and face unity.

* update english content

* [Android]fix ci bug.

* [Windows]adjust media recorder api.

* support pip i420

* update new sdk

* update exportPlist file

* [Android]fix token/app id input bug.

* [Android]fix push timestamp bug(NMS-12801)

* [Android]fix sensetime sticker oritation bug.

* enable filters

* fix custom capturer audio crash bug

* fix mutil capturer video bug

* fix custom video capturer publish streaming bug

* [Android]fix video filter invalid bug.

* [Android]remove publishCameraTrack=false

* [Windows]fix beauty external bug.

* Adapt to new versions

* [Windows]use timestamp of getCurrentMonotonicTimeInMs api to pushVideoFrame.

* update sdk version to 4.2.0

* [Windows]update sdk link.

* update azure ci

---------

Co-authored-by: zhaoyongqiang <zhaoyongqiang@agora.io>
Co-authored-by: Qianze Zhang <zhangqianze@agora.io>

.github/ci/build/build_ios_ipa.sh

@@ -17,10 +17,8 @@ TARGET_NAME=${PROJECT_PATH##*/}
 
 KEYCENTER_PATH=${PROJECT_PATH}"/"${TARGET_NAME}"/Common/KeyCenter.swift"
 
-METHOD_PATH=${PROJECT_PATH}"/ExportOptions.plist"
-
 # 打包环境
-CONFIGURATION=$method
+CONFIGURATION="Debug"
 
 #工程文件路径
 APP_PATH="${PROJECT_PATH}/${TARGET_NAME}.xcworkspace"
@@ -65,8 +63,6 @@ echo PBXPROJ_PATH: $PBXPROJ_PATH
 # Release
 /usr/libexec/PlistBuddy -c "Set :objects:03D13BF82448758C00B599B3:buildSettings:CURRENT_PROJECT_VERSION ${BUILD_NUMBER}" $PBXPROJ_PATH
 
-#修改打包方式
-/usr/libexec/PlistBuddy -c "Set :method $CONFIGURATION" $METHOD_PATH
 
 # 读取APPID环境变量
 echo AGORA_APP_ID:$APP_ID
@@ -87,33 +83,39 @@ xcodebuild clean -workspace "${APP_PATH}" -configuration "${CONFIGURATION}" -sch
 CURRENT_TIME=$(date "+%Y-%m-%d %H-%M-%S")
 
 # 归档路径
-ARCHIVE_PATH="${PROJECT_PATH}/${TARGET_NAME} ${CURRENT_TIME}/${TARGET_NAME}.xcarchive"
+ARCHIVE_PATH="${WORKSPACE}/${TARGET_NAME}_${BUILD_NUMBER}.xcarchive"
 # 编译环境
 
-# 导出路径
-EXPORT_PATH="${PROJECT_PATH}/${TARGET_NAME} ${CURRENT_TIME}"
-
 # plist路径
 PLIST_PATH="${PROJECT_PATH}/ExportOptions.plist"
 
 echo PLIST_PATH: $PLIST_PATH
 
 # archive 这边使用的工作区间 也可以使用project
-xcodebuild archive -workspace "${APP_PATH}" -scheme "${TARGET_NAME}" -configuration "${CONFIGURATION}" -archivePath "${ARCHIVE_PATH}" -destination 'generic/platform=iOS'
+xcodebuild CODE_SIGN_STYLE="Manual" archive -workspace "${APP_PATH}" -scheme "${TARGET_NAME}" clean CODE_SIGNING_REQUIRED=NO CODE_SIGNING_ALLOWED=NO -configuration "${CONFIGURATION}" -archivePath "${ARCHIVE_PATH}" -destination 'generic/platform=iOS' -quiet || exit
+
+cd ${WORKSPACE}
 
-# 导出ipa
-xcodebuild -exportArchive -archivePath "${ARCHIVE_PATH}" -exportPath "${EXPORT_PATH}" -exportOptionsPlist "${PLIST_PATH}"
+# 压缩archive
+7za a -tzip "${TARGET_NAME}_${BUILD_NUMBER}.xcarchive.zip" "${ARCHIVE_PATH}"
+
+# 签名
+# sh sign "${TARGET_NAME}_${BUILD_NUMBER}.xcarchive.zip" --type xcarchive --plist "${PLIST_PATH}"
+sh export "${TARGET_NAME}_${BUILD_NUMBER}.xcarchive.zip" --plist "${PLIST_PATH}"
 
 # 上传IPA
-7za a "$WORKSPACE/${TARGET_NAME}_${BUILD_NUMBER}_IPA.zip" -r "${EXPORT_PATH}/${TARGET_NAME}.ipa"
+PAYLOAD_PATH="${TARGET_NAME}_${BUILD_NUMBER}_Payload"
+mkdir "${PAYLOAD_PATH}"
+# mv "${TARGET_NAME}_${BUILD_NUMBER}_iOS.ipa" "${PAYLOAD_PATH}"
+mv "${TARGET_NAME}_${BUILD_NUMBER}.ipa" "${PAYLOAD_PATH}"
 
-# 删除IPA文件夹
-rm -rf "${EXPORT_PATH}"
+7za a "${TARGET_NAME}_${BUILD_NUMBER}_IPA.zip" -r "${PAYLOAD_PATH}"
+python3 artifactory_utils.py --action=upload_file --file="${TARGET_NAME}_${BUILD_NUMBER}_IPA.zip" --project
 
-# rm -rf "${EXPORT_PATH}/${TARGET_NAME}.xcarchive"
-# rm -rf "${EXPORT_PATH}/Packaging.log"
-# rm -rf "${EXPORT_PATH}/ExportOptions.plist"
-# rm -rf "${EXPORT_PATH}/DistributionSummary.plist"
+# 删除IPA文件夹
+rm -rf ${TARGET_NAME}_${BUILD_NUMBER}.xcarchive
+rm -rf *.zip
+rm -rf ${PAYLOAD_PATH}
 
 #复原Keycenter文件
 python3 /tmp/jenkins/api-examples/.github/ci/build/modify_ios_keycenter.py $KEYCENTER_PATH 1

.github/ci/build/build_mac.sh

@@ -71,7 +71,7 @@ else
     echo "failed"
     exit 1
 fi
-cp -rf ./macOS/** ./$unzip_name/samples/APIExample
+cp -a ./macOS/** ./$unzip_name/samples/APIExample 
 mv ./$unzip_name/samples/APIExample/sdk.podspec ./$unzip_name/
 python3 ./.github/ci/build/modify_podfile.py ./$unzip_name/samples/APIExample/Podfile
 

.github/ci/build/build_mac_ipa.sh

@@ -17,7 +17,7 @@ TARGET_NAME=${PROJECT_PATH##*/}
 KEYCENTER_PATH=${PROJECT_PATH}"/"${TARGET_NAME}"/Common/KeyCenter.swift"
 
 # 打包环境
-CONFIGURATION=developer-id 
+CONFIGURATION=Release
 
 #工程文件路径
 APP_PATH="${PROJECT_PATH}/${TARGET_NAME}.xcworkspace"
@@ -69,12 +69,9 @@ xcodebuild clean -workspace "${APP_PATH}" -configuration "${CONFIGURATION}" -sch
 CURRENT_TIME=$(date "+%Y-%m-%d %H-%M-%S")
 
 # 归档路径
-ARCHIVE_PATH="${PROJECT_PATH}/${TARGET_NAME} ${CURRENT_TIME}/${TARGET_NAME}.xcarchive"
+ARCHIVE_PATH="${WORKSPACE}/${TARGET_NAME}_${BUILD_NUMBER}.xcarchive"
 # 编译环境
 
-# 导出路径
-EXPORT_PATH="${PROJECT_PATH}/${TARGET_NAME} ${CURRENT_TIME}"
-
 # plist路径
 PLIST_PATH="${PROJECT_PATH}/ExportOptions.plist"
 
@@ -83,20 +80,20 @@ echo PLIST_PATH: $PLIST_PATH
 # archive 这边使用的工作区间 也可以使用project
 xcodebuild archive -workspace "${APP_PATH}" -scheme "${TARGET_NAME}" -configuration "${CONFIGURATION}" -archivePath "${ARCHIVE_PATH}"
 
-# 导出ipa
-xcodebuild -exportArchive -archivePath "${ARCHIVE_PATH}" -exportPath "${EXPORT_PATH}" -exportOptionsPlist "${PLIST_PATH}"
+cd ${WORKSPACE}
 
-# 删除archive文件
-rm -rf "${EXPORT_PATH}/${TARGET_NAME}.xcarchive"
-rm -rf "${EXPORT_PATH}/Packaging.log"
-rm -rf "${EXPORT_PATH}/ExportOptions.plist"
-rm -rf "${EXPORT_PATH}/DistributionSummary.plist"
+# 压缩archive
+7za a -slp "${TARGET_NAME}_${BUILD_NUMBER}.xcarchive.zip" "${ARCHIVE_PATH}"
+
+# 签名
+sh sign "${WORKSPACE}/${TARGET_NAME}_${BUILD_NUMBER}.xcarchive.zip" --type xcarchive --plist "${PLIST_PATH}" --application macApp
 
 # 上传IPA
-7za a "$WORKSPACE/${TARGET_NAME}_Mac_${BUILD_NUMBER}_APP.zip" -r "${EXPORT_PATH}"
+python3 artifactory_utils.py --action=upload_file --file="${TARGET_NAME}_${BUILD_NUMBER}.app.zip" --project
 
-# 删除IPA文件夹
-rm -rf "${EXPORT_PATH}"
+# 删除archive文件
+rm -rf ${TARGET_NAME}_${BUILD_NUMBER}.xcarchive
+rm -rf *.zip
 
 #复原Keycenter文件
 python3 /tmp/jenkins/api-examples/.github/ci/build/modify_ios_keycenter.py $KEYCENTER_PATH 1

Android/APIExample-Audio/app/build.gradle

@@ -48,7 +48,7 @@ dependencies {
         implementation fileTree(dir: "${localSdkPath}", include: ['*.jar', '*.aar'])
     }
     else{
-        def agora_sdk_version = "4.1.1"
+        def agora_sdk_version = "4.2.0"
         // case 1: full single lib with voice only
         implementation "io.agora.rtc:voice-sdk:${agora_sdk_version}"
         // case 2: partial libs with voice only

Android/APIExample-Audio/app/src/main/java/io/agora/api/example/examples/advanced/VoiceEffects.java

@@ -27,16 +27,27 @@
 import static io.agora.rtc2.Constants.ULTRA_HIGH_QUALITY_VOICE;
 import static io.agora.rtc2.Constants.VOICE_BEAUTIFIER_OFF;
 import static io.agora.rtc2.Constants.VOICE_CHANGER_BASS;
+import static io.agora.rtc2.Constants.VOICE_CHANGER_CARTOON;
+import static io.agora.rtc2.Constants.VOICE_CHANGER_CHILDLIKE;
+import static io.agora.rtc2.Constants.VOICE_CHANGER_CHIPMUNK;
+import static io.agora.rtc2.Constants.VOICE_CHANGER_DARTH_VADER;
 import static io.agora.rtc2.Constants.VOICE_CHANGER_EFFECT_BOY;
 import static io.agora.rtc2.Constants.VOICE_CHANGER_EFFECT_GIRL;
 import static io.agora.rtc2.Constants.VOICE_CHANGER_EFFECT_HULK;
 import static io.agora.rtc2.Constants.VOICE_CHANGER_EFFECT_OLDMAN;
 import static io.agora.rtc2.Constants.VOICE_CHANGER_EFFECT_PIGKING;
 import static io.agora.rtc2.Constants.VOICE_CHANGER_EFFECT_SISTER;
 import static io.agora.rtc2.Constants.VOICE_CHANGER_EFFECT_UNCLE;
+import static io.agora.rtc2.Constants.VOICE_CHANGER_GIRLISH_MAN;
+import static io.agora.rtc2.Constants.VOICE_CHANGER_GROOT;
+import static io.agora.rtc2.Constants.VOICE_CHANGER_IRON_LADY;
+import static io.agora.rtc2.Constants.VOICE_CHANGER_MONSTER;
 import static io.agora.rtc2.Constants.VOICE_CHANGER_NEUTRAL;
+import static io.agora.rtc2.Constants.VOICE_CHANGER_PHONE_OPERATOR;
+import static io.agora.rtc2.Constants.VOICE_CHANGER_SHIN_CHAN;
 import static io.agora.rtc2.Constants.VOICE_CHANGER_SOLID;
 import static io.agora.rtc2.Constants.VOICE_CHANGER_SWEET;
+import static io.agora.rtc2.Constants.VOICE_CHANGER_TRANSFORMERS;
 import static io.agora.rtc2.Constants.VOICE_CONVERSION_OFF;
 
 import android.content.Context;
@@ -97,7 +108,7 @@ public class VoiceEffects extends BaseFragment implements View.OnClickListener,
             chatBeautifier, timbreTransformation, voiceChanger, styleTransformation, roomAcoustics, pitchCorrection, _pitchModeOption, _pitchValueOption, voiceConversion,
             customBandFreq, customReverbKey;
     private ViewGroup _voice3DLayout, _pitchModeLayout, _pitchValueLayout;
-    private SeekBar _voice3DCircle, customPitch, customBandGain, customReverbValue;
+    private SeekBar _voice3DCircle, customPitch, customBandGain, customReverbValue, customVoiceFormant;
 
     private AudioSeatManager audioSeatManager;
 
@@ -159,10 +170,12 @@ public void onViewCreated(@NonNull View view, @Nullable Bundle savedInstanceStat
         customBandGain = view.findViewById(R.id.audio_custom_band_gain); // engine.setLocalVoiceEqualization()
         customReverbKey = view.findViewById(R.id.audio_custom_reverb_key);
         customReverbValue = view.findViewById(R.id.audio_custom_reverb_value); //engine.setLocalVoiceReverb()
+        customVoiceFormant = view.findViewById(R.id.audio_voice_formant_value); //engine.setLocalVoiceFormant()
 
         customPitch.setOnSeekBarChangeListener(this);
         customBandGain.setOnSeekBarChangeListener(this);
         customReverbValue.setOnSeekBarChangeListener(this);
+        customVoiceFormant.setOnSeekBarChangeListener(this);
         customBandFreq.setOnItemSelectedListener(this);
         customReverbKey.setOnItemSelectedListener(this);
 
@@ -194,6 +207,7 @@ private void resetControlLayoutByJoined() {
         customBandGain.setEnabled(joined);
         customReverbKey.setEnabled(joined);
         customReverbValue.setEnabled(joined);
+        customVoiceFormant.setEnabled(joined);
 
 
         chatBeautifier.setSelection(0);
@@ -206,6 +220,7 @@ private void resetControlLayoutByJoined() {
         customPitch.setProgress(0);
         customBandGain.setProgress(0);
         customReverbValue.setProgress(0);
+        customVoiceFormant.setProgress(50);
     }
 
     @Override
@@ -407,8 +422,8 @@ private void joinChannel(String channelId) {
     private final IRtcEngineEventHandler iRtcEngineEventHandler = new IRtcEngineEventHandler() {
         /**
          * Error code description can be found at:
-         * en: https://api-ref.agora.io/en/voice-sdk/android/4.x/API/class_irtcengineeventhandler.html#callback_irtcengineeventhandler_onerror
-         * cn: https://docs.agora.io/cn/voice-call-4.x/API%20Reference/java_ng/API/class_irtcengineeventhandler.html#callback_irtcengineeventhandler_onerror
+         * en: https://api-ref.agora.io/en/video-sdk/android/4.x/API/class_irtcengineeventhandler.html#callback_irtcengineeventhandler_onerror
+         * cn: https://docs.agora.io/cn/video-call-4.x/API%20Reference/java_ng/API/class_irtcengineeventhandler.html#callback_irtcengineeventhandler_onerror
          */
         @Override
         public void onError(int err) {
@@ -617,6 +632,28 @@ private int getVoiceConversionValue(String label) {
                 return VOICE_CHANGER_SOLID;
             case "VOICE_CHANGER_BASS":
                 return VOICE_CHANGER_BASS;
+            case "VOICE_CHANGER_CARTOON":
+                return VOICE_CHANGER_CARTOON;
+            case "VOICE_CHANGER_CHILDLIKE":
+                return VOICE_CHANGER_CHILDLIKE;
+            case "VOICE_CHANGER_PHONE_OPERATOR":
+                return VOICE_CHANGER_PHONE_OPERATOR;
+            case "VOICE_CHANGER_MONSTER":
+                return VOICE_CHANGER_MONSTER;
+            case "VOICE_CHANGER_TRANSFORMERS":
+                return VOICE_CHANGER_TRANSFORMERS;
+            case "VOICE_CHANGER_GROOT":
+                return VOICE_CHANGER_GROOT;
+            case "VOICE_CHANGER_DARTH_VADER":
+                return VOICE_CHANGER_DARTH_VADER;
+            case "VOICE_CHANGER_IRON_LADY":
+                return VOICE_CHANGER_IRON_LADY;
+                case "VOICE_CHANGER_SHIN_CHAN":
+                return VOICE_CHANGER_SHIN_CHAN;
+            case "VOICE_CHANGER_GIRLISH_MAN":
+                return VOICE_CHANGER_GIRLISH_MAN;
+            case "VOICE_CHANGER_CHIPMUNK":
+                return VOICE_CHANGER_CHIPMUNK;
             case "VOICE_CONVERSION_OFF":
             default:
                 return VOICE_CONVERSION_OFF;
@@ -743,6 +780,9 @@ public void onCheckedChanged(CompoundButton buttonView, boolean isChecked) {
 
     @Override
     public void onProgressChanged(SeekBar seekBar, int progress, boolean fromUser) {
+        if (!fromUser) {
+            return;
+        }
         if(seekBar == _voice3DCircle){
             int cicle = (int) (1 + 59 * progress * 1.0f / seekBar.getMax());
             // [1,60], 10 default
@@ -771,6 +811,10 @@ public void onProgressChanged(SeekBar seekBar, int progress, boolean fromUser) {
                 value = (int) (100 * progress * 1.0f / seekBar.getMax());
             }
             engine.setLocalVoiceReverb(reverbKey, value);
+        } else if (seekBar == customVoiceFormant) {
+            // [-1, 1]
+            double value = (progress - 50) * 1.0f / 100;
+            engine.setLocalVoiceFormant(value);
         }
     }
 

Android/APIExample-Audio/app/src/main/java/io/agora/api/example/examples/advanced/customaudio/CustomAudioRender.java

@@ -236,6 +236,7 @@ public void onClick(View v) {
                 engine.leaveChannel();
                 pulling = false;
                 join.setText(getString(R.string.join));
+                audioSeatManager.downAllSeats();
                 if(pullingTask != null){
                     try {
                         pullingTask.join();
@@ -268,7 +269,7 @@ private void joinChannel(String channelId) {
          *   0: Success.
          *   < 0: Failure.
          * PS: Ensure that you call this method before the joinChannel method.*/
-        engine.setExternalAudioSource(true, SAMPLE_RATE, SAMPLE_NUM_OF_CHANNEL, 2, false, true);
+        // engine.setExternalAudioSource(true, SAMPLE_RATE, SAMPLE_NUM_OF_CHANNEL, 2, false, true);
 
 
 
@@ -303,9 +304,8 @@ private void joinChannel(String channelId) {
     private final IRtcEngineEventHandler iRtcEngineEventHandler = new IRtcEngineEventHandler() {
 
         /**
-         * Error code description can be found at:
-         * en: https://api-ref.agora.io/en/voice-sdk/android/4.x/API/class_irtcengineeventhandler.html#callback_irtcengineeventhandler_onerror
-         * cn: https://docs.agora.io/cn/voice-call-4.x/API%20Reference/java_ng/API/class_irtcengineeventhandler.html#callback_irtcengineeventhandler_onerror
+         * en: https://api-ref.agora.io/en/video-sdk/android/4.x/API/class_irtcengineeventhandler.html#callback_irtcengineeventhandler_onerror
+         * cn: https://docs.agora.io/cn/video-call-4.x/API%20Reference/java_ng/API/class_irtcengineeventhandler.html#callback_irtcengineeventhandler_onerror
          */
         @Override
         public void onError(int err) {