AVAudioEngine在macOS/iOS上协调/同步输入/输出时间戳

我正在尝试将录制的音频（从AVAudioEngine inputNode）同步到录制过程中正在播放的音频文件。结果应该类似于多轨录音，其中每个后续新音轨都与录制时正在播放的先前音轨同步。

因为sampleTime在不同之间AVAudioEngine的输出和输入节点，我使用hostTime确定偏移原始音频和输入缓冲器。

在 iOS 上，我会假设我必须使用AVAudioSession的各种延迟属性 ( inputLatency, outputLatency, ioBufferDuration) 来协调轨道以及主机时间偏移，但我还没有想出使它们工作的神奇组合。这同样适用于各种AVAudioEngine和Node属性，如latency和presentationLatency.

在 macOS 上，AVAudioSession不存在（在 Catalyst 之外），这意味着我无法访问这些数字。同时，latency/presentationLatency属性在大多数情况下会AVAudioNodes报告0.0。在 macOS 上，我确实可以访问AudioObjectGetPropertyData并且可以向系统询问kAudioDevicePropertyLatency, kAudioDevicePropertyBufferSize、kAudioDevicePropertySafetyOffset等，但是对于协调所有这些的公式又有点不知所措。

我在https://github.com/jnpdx/AudioEngineLoopbackLatencyTest上有一个示例项目，它运行一个简单的环回测试（在 macOS、iOS 或 Mac Catalyst 上）并显示结果。在我的 Mac 上，轨道之间的偏移量约为 720 个样本。在其他人的 Mac 上，我看到多达 1500 个样本偏移。

在我的 iPhone 上，我可以通过使用AVAudioSession's outputLatency+使它接近完美的样本inputLatency。然而，同样的公式让我的 iPad 上的东西没有对齐。

在每个平台上同步输入和输出时间戳的神奇公式是什么？我知道每个人可能会有所不同，这很好，而且我知道我不会获得 100% 的准确度，但我想在进行我自己的校准过程之前尽可能接近

这是我当前代码的示例（完整的同步逻辑可以在https://github.com/jnpdx/AudioEngineLoopbackLatencyTest/blob/main/AudioEngineLoopbackLatencyTest/AudioManager.swift找到）：

//Schedule playback of original audio during initial playback
let delay = 0.33 * state.secondsToTicks
let audioTime = AVAudioTime(hostTime: mach_absolute_time() + UInt64(delay))
state.audioBuffersScheduledAtHost = audioTime.hostTime

...

//in the inputNode's inputTap, store the first timestamp
audioEngine.inputNode.installTap(onBus: 0, bufferSize: 1024, format: recordingFormat) { (pcmBuffer, timestamp) in
            if self.state.inputNodeTapBeganAtHost == 0 {
                self.state.inputNodeTapBeganAtHost = timestamp.hostTime
            }
}

...

//after playback, attempt to reconcile/sync the timestamps recorded above

let timestampToSyncTo = state.audioBuffersScheduledAtHost
let inputNodeHostTimeDiff = Int64(state.inputNodeTapBeganAtHost) - Int64(timestampToSyncTo)
let inputNodeDiffInSamples = Double(inputNodeHostTimeDiff) / state.secondsToTicks * inputFileBuffer.format.sampleRate //secondsToTicks is calculated using mach_timebase_info

//play the original metronome audio at sample position 0 and try to sync everything else up to it
let originalAudioTime = AVAudioTime(sampleTime: 0, atRate: renderingEngine.mainMixerNode.outputFormat(forBus: 0).sampleRate)
originalAudioPlayerNode.scheduleBuffer(metronomeFileBuffer, at: originalAudioTime, options: []) {
  print("Played original audio")
}

//play the tap of the input node at its determined sync time -- this _does not_ appear to line up in the result file
let inputAudioTime = AVAudioTime(sampleTime: AVAudioFramePosition(inputNodeDiffInSamples), atRate: renderingEngine.mainMixerNode.outputFormat(forBus: 0).sampleRate)
recordedInputNodePlayer.scheduleBuffer(inputFileBuffer, at: inputAudioTime, options: []) {
  print("Input buffer played")
}

运行示例应用程序时，这是我得到的结果：

回答

// Some non-zero value to get AVAudioEngine running
let startDelay = 0.1

// The original audio file start time
let originalStartingFrame: AVAudioFramePosition = AVAudioFramePosition(playerNode.outputFormat(forBus: 0).sampleRate * startDelay)

// The output tap's first sample is delivered to the device after the buffer is filled once
// A number of zero samples equal to the buffer size is produced initially
let outputStartingFrame: AVAudioFramePosition = Int64(state.outputBufferSizeFrames)

// The first output sample makes it way back into the input tap after accounting for all the latencies
let inputStartingFrame: AVAudioFramePosition = outputStartingFrame - Int64(state.outputLatency + state.outputStreamLatency + state.outputSafetyOffset + state.inputSafetyOffset + state.inputLatency + state.inputStreamLatency)

// Output:
// kAudioDevicePropertySafetyOffset:    144
// kAudioDevicePropertyLatency:          11
// kAudioStreamPropertyLatency:         424
// kAudioDevicePropertyBufferFrameSize: 512

// Input:
// kAudioDevicePropertySafetyOffset:     154
// kAudioDevicePropertyLatency:            0
// kAudioStreamPropertyLatency:         2404
// kAudioDevicePropertyBufferFrameSize:  512

originalStartingFrame =  4410
outputStartingFrame   =   512
inputStartingFrame    = -2625