Music Video - syncing audio to overcranked video

[jcrhee@mac.com]

Hopefully somebody can help me figure this out. I'm going to be shooting a music video on 35mm. For one of the shots that I'm thinking, I want to overcrank the camera at 32fps while the singer is singing so that when played back at 24fps, the singer's movement is slower but his mouth will sync with the real time music. Is there a logical formula or guideline for figuring out how fast I should have the audio playback at so that it will sync up perfectly. Thanks

[Jerry_R]

Assuming that you are doing separate sound you should check out:
http://trewaudio.com/store/product.p...1&cat=3&page=1
but if those techniques do not apply you will need to fix it in post which is going to be time consuming and expensive. You will need a time strech/compress that preserves both pitch and formants. Waves, TCElectronics power core and audio box all are good for tis but not cheap. You will need to strech/compress and slide tracks in post so accurate track sheets and TC stamps are imperative.

You are not taking an easy or simple track here and its going to be expensive.

[Sam_Adj]

just speed up the audio by 1.33 or 1.5 if you are overcrankin' 36fps, and it is better to maintain the pitch so the singer can memorize the speed and can sing along with the fast audio, anyhow, i usualy tell the singers to sing loudly so they can perform better and maintain good lip-synch..

and if it is going to be long shots, you better be as acurate as you can and i suggest you recored the speeded up audio on DAT since it is more acurate than normal CD players... but if it is going to be fastly edited (short shots) then less acuracy required.

at last, since it is going to be slowed down in post, don't worry too much because slow lipsynch and movmets help hiding some small flaws and slips..

[Jay Rose]

Is there a logical formula or guideline for figuring out how fast I should have the audio playback at so that it will sync up perfectly.

Of course there is. 24:32 is (when divided by 8) a slowdown of 3:4, or 1.333%.

Less logical is how you're going to do the job. Making something 33% slower without affecting the pitch can be difficult. The slowdown part is easy, but pitch correction that radical is going to involve a lot of stutters*: plan on making multiple passes with different starting times, and take the best of each.

The best tool for that big a slowdown is a phase vocoder, which can stretch the voice without stutters, but it'll only work well on a solo voice track. Then you can use a more conventional pitch shifter on the instrumentals... but watch for flutters on the percussion that turn into drum flams. (If the instrumentals were done on a MIDI sequencer, you're in luck: go back to the original files and change the tempo while the key stays the same.)

Or consider, for just this effects shot, having the singer do a new slower vocal to a slowed-down instrumental. Add LOTS of verb in the mix...

---
* A slow down with conventional software requires repeating very small pieces of audio. Today's programs are pretty good at looping those tiny pieces without noticeable splices... until you get into very quick stop consonants like /t/ and /d/. Then it's just a question of luck whether a critical part of a consonant will be noticeably repeated.

Jerry, I'm not sure formant correction is needed here, since we're coming back to the original performed pitch and just changing the timing. The formant resonances shouldn't be changing. But maybe I'm not seeing this right.

---

It used to be a lot harder. I did a short film track in 1978 that had a lot of speed changes (through step printing) on lipsync scenes. We were able to squeak by with the only tool practical at the time - an Eventide 910 - but I had to invent a pseudo pilottone with a mid-frequency generator, and then resolve by looking at a frequency counter.

[Jerry_R]

Jay most of the pitch shifters seem to be aimed at instruments only a few at voice correction, and even fewer at general use. Those can change formants on large shifts like this.
But on reconsidering I think it will be easier and cheaper to have them mime and lipsync to the slow version on video and then just cut it in in post.

[Sam_Adj]

John, Jerry, am i missing something in here?? why to slowdown the audio?? as it is overcranking, you must speed up the audio track so the lipsynch will retain normal after slowing down the film by 1.333 factor, beside, here the critical thing is the speed that most be acurate not the quality as the speeded up audio won't be used for anything after the shooting..!!!

[Jerry_R]

Actually they would sync to the sped up version and then it should work with the slowed down version.

[Sam_Adj]

Why the slowed down version?? you mean the normal speed audio version to be synched in post with the slowed film.. then why to speed down audio?

[Jerry_R]

They play to the sped up version. And as it will never be used except as a guide track how its sped up doesn't matter so long as it is intelligible to the band and singer. Now when you slow the material to 24fps you use the original version.

When I reread that last post it didn't make sense to me either.

[Jay Rose]

Sort of depends on the goal.

I read it as the OP wanted a couple of scenes to looks super slowed down, with stretched out sounds to match.

If you shoot at 32 fps to an audio guide that's running 33% fast, I should think the result would very sloppy sync when projected at 24fps... the singers would have to lip much faster than normal - like a patter song - and then on playback any places that they missed lipping (inevitable at that speed) would be magnified. But maybe that's what the OP wanted.

Then again, if it's only a couple of scenes, maybe the goal is to have dreamlike motions without disrupting the tempo of the song. In that case, if the band is making big movements - like jumping up and down - it would require fairly wide shots and the errors of 'patter-song lipping' wouldn't be that obvious. And for short scenes, you could fudge... could be an interesting effect.

In that case, you might consider having the singers lip to a click track, rather than a playback at 33% fast, so you don't have to worry about pitch shifting at all.

[Sam_Adj]

In fact Jay you are right about the motions in the slo-mo scene, i tested once fast lipsynchin' i was usiing DigiBeta, i sped up the sound by two since the best slo-mo for video would be 200% and i realy didnt like what i had, the problem is, if you dont have any live or fast organic-like motions in the scene (like flying hair or cloths, dancing people or birds maybe), you will end up with the singer singing at the normal speed but making weird faces

[jcrhee@mac.com]

To clarify somethings, We are going to be shooting on 35mm film which runs at 24fps. What I'm planning on doing is overcranking it to 32fps so that when played back at 24, it will be in slow motion. That slow motion needs to sync up to the audio track which will be playing back in real time. So inorder for that to work, the audio will need to play back faster during filming. Now I thought that 1.33% faster is the right info for the speed of the music but we did a test on my HVX - overcranked to 32fps and converted down to 24fps in Final Cut 5.1.2. THe slowmo video and the real-time audio did not match up. The video was playing back slower. Just for reference, look up Coldplay's Yellow music video on YouTube. Now they shot that at 48fps on 35mm, when played back in real time (24fps), the footage is 2x slower than real-time. Which means that the music playback during filming has to be playing back 2x faster for it to match up. Now looking at 32fps, in relation to 24fps, it is playing back at 75% speed. So does this mean that the audio should be playing back at 150% faster. And is this audio when played at 75%, equal to real-time or normal speed of the original music?

[greg_m]

Nope, not 150%. Jay's original answer was right, it is 133% (actually 133.33... %).

You may be confused about the math. You said "1.33% faster" but that is very different from 133%. "1.33% faster" means 1.33% faster than 100%, or 101.33%.

If your program actually uses the terms "faster than" or something like that, you actually want to be 33% faster than normal!

You could double check this in advance, by looking at the length of the track. The length of the speeded-up guide track should be 75% the length of the original track. For example, if you're syncing to a song that is 2:17.35 long, that equals 137.35 seconds. The speeded up version should be 75% as long; that's 103.01 seconds, or 1:43.01.

If you film 103.01 seconds of action at 32fps, that's 3296 frames. When you project that at 24fps, it will project for 137.35 seconds, which is the length of the original music clip.

Make sense now?

[Jay Rose]

we did a test on my HVX - overcranked to 32fps and converted down to 24fps in Final Cut 5.1.2. THe slowmo video and the real-time audio did not match up

If you saw a small drift, suspect the test method: a DV camera's idea of 32 fps and 24fps might not be precise. That's both because of the need to be eventually compatible with NTSC without having too-convoluted DSP, and because... well, manufacturers can find it slightly cheaper to cheat (remember Canon's infamous not-quite 48 kHz sampling?).

The OP specified film, which is a different animal. Film cameras can shoot at precise speeds (to the limits of their crystals). But then, of course, nobody's mentioned the pulldown which will be necessary before editing in a video environment... if it's just a few quick cuts, the sync error because of pulldown will be negligible and can be tweaked at each edit. But if it's the whole song, that's got to be figured as well.

[jcrhee@mac.com]

Thanks for the help. By the way 1.33% was a typo, it should have been 133%. I should suspect the DV camera. I'm going to find out what's the relationship between HVX's 32fps and finl 32fps. Thanks for the help.