使用 AVCaptureSession 和 AVAssetWriter 翻转相机时无缝录音

我正在寻找一种在前后摄像头之间切换时保持无缝音轨的方法。市场上的许多应用程序都可以做到这一点，一个例子是 SnapChat……

解决方案应使用 AVCaptureSession 和 AVAssetWriter。此外，它不应该明确使用 AVMutableComposition，因为有一个bugAVMutableComposition 和 AVCaptureSession ATM 之间。另外，我负担不起后期处理时间。

目前，当我更改视频输入时，音频录制会跳过并变得不同步。

我添加了可能相关的代码。

翻转相机

-(void) updateCameraDirection:(CamDirection)vCameraDirection {
    if(session) {
        AVCaptureDeviceInput* currentInput;
        AVCaptureDeviceInput* newInput;
        BOOL videoMirrored = NO;
        switch (vCameraDirection) {
            case CamDirection_Front:
                currentInput = input_Back;
                newInput = input_Front;
                videoMirrored = NO;
                break;
            case CamDirection_Back:
                currentInput = input_Front;
                newInput = input_Back;
                videoMirrored = YES;
                break;
            default:
                break;
        }

        [session beginConfiguration];
        //disconnect old input
        [session removeInput:currentInput];
        //connect new input
        [session addInput:newInput];
        //get new data connection and config
        dataOutputVideoConnection = [dataOutputVideo connectionWithMediaType:AVMediaTypeVideo];
        dataOutputVideoConnection.videoOrientation = AVCaptureVideoOrientationPortrait;
        dataOutputVideoConnection.videoMirrored = videoMirrored;
        //finish
        [session commitConfiguration];
    }
}

样品缓冲液

- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection {
    //not active
    if(!recordingVideo)
        return;

    //start session if not started
    if(!startedSession) {
        startedSession = YES;
        [assetWriter startSessionAtSourceTime:CMSampleBufferGetPresentationTimeStamp(sampleBuffer)];
    }

    //Process sample buffers
    if (connection == dataOutputAudioConnection) {
        if([assetWriterInputAudio isReadyForMoreMediaData]) {
            BOOL success = [assetWriterInputAudio appendSampleBuffer:sampleBuffer];
            //…
        }

    } else if (connection == dataOutputVideoConnection) {
        if([assetWriterInputVideo isReadyForMoreMediaData]) {        
            BOOL success = [assetWriterInputVideo appendSampleBuffer:sampleBuffer];
            //…
        }
    }
}

也许调整音频采样时间戳？

嘿，我面临着同样的问题，发现切换相机后下一帧被推得很远。这似乎会改变之后的每一帧，从而导致视频和音频不同步。我的解决方案是在切换相机后将每个错位的帧移动到正确的位置。

抱歉，我的答案将在 Swift 4.2 中

你必须使用AVAssetWriterInputPixelBufferAdaptor为了在指定的呈现时间戳处附加示例缓冲区。

previousPresentationTimeStamp是前一帧的呈现时间戳，currentPresentationTimestamp正如您所猜测的当前的呈现时间戳。maxFrameDistance测试时效果很好，但您可以根据自己的喜好进行更改。

let currentFramePosition = (Double(self.frameRate) * Double(currentPresentationTimestamp.value)) / Double(currentPresentationTimestamp.timescale)
let previousFramePosition = (Double(self.frameRate) * Double(previousPresentationTimeStamp.value)) / Double(previousPresentationTimeStamp.timescale)
var presentationTimeStamp = currentPresentationTimestamp
let maxFrameDistance = 1.1
let frameDistance = currentFramePosition - previousFramePosition
if frameDistance > maxFrameDistance {
    let expectedFramePosition = previousFramePosition + 1.0
    //print("[mwCamera]: Frame at incorrect position moving from \(currentFramePosition) to \(expectedFramePosition)")

    let newFramePosition = ((expectedFramePosition) * Double(currentPresentationTimestamp.timescale)) / Double(self.frameRate)

    let newPresentationTimeStamp = CMTime.init(value: CMTimeValue(newFramePosition), timescale: currentPresentationTimestamp.timescale)

    presentationTimeStamp = newPresentationTimeStamp
}

let success = assetWriterInputPixelBufferAdator.append(pixelBuffer, withPresentationTime: presentationTimeStamp)
if !success, let error = assetWriter.error {
    fatalError(error.localizedDescription)
}

另请注意- 这之所以有效，是因为我保持了帧速率一致，因此请确保您在整个过程中完全控制捕获设备的帧速率。

我这里有一个使用这个逻辑的仓库