How do you convert an arbitrary video file into a playable Flash video using freely available programs and methods? After close to an afternoon of searching, testing and head-scratching, I finally have a whole answer that can be applied ad-hoc to almost any video you can get your hands on.
This "guide" (more anecdotal than how-to) assumes knowledge of video encoding basics, I'm not going to cover the difference between container and video formats or how to use VirtualDub, there are plenty of other tutorials and guides that cover those topics.
The Flash Video container format (FLV) supports two major video formats: the first is codenamed "Sorenson Spark" and is a variant on the H.263 standard; the second is "On2 TrueMotion VP6". The former is well supported in many encoding and decoding tools and libraries, the latter isn't. It will come as no surprise then that the latter allows for far greater compression at similar visual quality when compared apples-to-apples to the H.263 format. To give you a quantifiable measure of this: I managed to get more than 2 times greater compression and better visual quality when using the VP6 codec. In short, this is the codec you want to use to get the most out of your movies.
Quick and dirty
If you just want to get a compatible FLV file quickly and painlessly and aren't worried about size or quality overmuch, then grab yourself a copy of ffmpeg (a recent compiled Windows binary can be found here) and put it in a place where your command-line of choice can find it. Then punch in:
ffmpeg -i "yourvideofilegoeshere.avi" outfile.flv
Voila, in no time flat you'll have an all-singing all-dancing .flv file ready for whatever you have in store for it. If you're feeling particularly awesome, you can even control the output size of the video:
ffmpeg -i "totallyawesomekittenvideo.avi" -s 320x240 outfile.flv
ffmpeg, converts using the H.263 video flavour and MP3 audio format which is probably fine for most people. The problem with this process is the output is less than stellar and suffers from a tremendous amount of artefacting. I wanted to exercise a little more control over the visual quality. Some searching revealed byzantine quantizer settings which made the command line look like a calculator had exploded:
ffmpeg -i "hahathatguytotallysucks.avi" -qcomp 0.6 -qmax 15 -qdiff 4 -i_qfactor 0.71428572 -b_qfactor 0.76923078 -maxrate 972800 -s 320x240 -b 819200 -refs 1 -subq 1 -y outfile.flv
Tweaking these options gives variable results, but nothing close to the kind of quality / file-size ratio I wanted. The VP6 codec seemed worth trying out. Unfortunately using the VP6 codec is rife with hurdles; the primary one is that it is entirely proprietary, On2 own licenses and patents and probably crocodiles with bazookas to protect the codec; some companies have obtained licenses to use it in their products (On2 of course having their own implementation) which means the easiest and most pain-free route is to buy one of those products and bask in the fully-licensed glory.
What isn't widely publicised is that On2 released a version of the VP6 codec for "Personal use" but no longer provide it for download on their website. A cursory search on Google (lets say "vp6 vfw codec") returns some good matches. After downloading and installing, I now had the ability to encode to VP6 as long as it's for "Personal use" according to the license agreement. This little endeavour was for my own curiosity rather than monetary gain which I'm sure falls under that stipulation.
Codec in hand, in theory it should be as simple as encoding to VP6 using something like VirtualDub and then muxing everything together into an FLV file. If only things were that simple. As far as I could see, there exists no standalone set of FLV muxing tools (like the seminal MKVtoolnix suite). However, ffmpeg can output to an FLV file and provides the ability to do a straight copy (i.e. no transcoding) of the source video, that could work...
No.
For the VP6 codec to be recognised within an FLV file, the container needs to have special bits set which indicate to the player that it's going to receive VP6 video content rather than H.263/Spark; ffmpeg doesn't write these bits as it doesn't "officially" deal with VP6. After much searching, I stumbled upon a way to modify ffmpeg to write these bits but the patch hasn't been merged into the main ffmpeg branch yet. You can download the patch and a pre-compiled Windows binary from a blog which also offers an alternate method of achieving what I'm describing.
The crux
If you haven't been keeping up, here's the short version of it. We want to make an FLV using VP6. Vanilla ffmpeg doesn't do this so we need a modified ffmpeg to do it. Use the above blog page or modify your ffmpeg source to get ffmpeg to do what we want.
Open up your source video in VirtualDub and apply any filters you want (contrast, brightness, resize etc.) but make sure the "Flip vertically" filter is somewhere in that mix. Go to compression and select the VP6 codec; if you're using VP 6.2 you can do two-pass encoding which, potentially gives better results than a one-pass encode. Make sure "Use source audio" is selected and save your video down. You'll have an .avi file which has VP6 video and the original source audio.
Now use your modified version of ffmpeg and use the following (substituting filenames where applicable):
ffmpeg -y -i "ooooohprettyprettyflowers.avi" -vcodec copy outfile.flv
If you want to control the audio compression a little better add the options for that:
ffmpeg -b 128 -ac 2 -ar 44100 -y -i "ohwowexplosions.avi" -vcodec copy outfile.flv
You will now have "outfile.flv" which is your final Flash video file, ready for uploading. Of course, the proof of the pudding is in the tasting:
h263.flv - 1,466KB
vp62.flv - 676KB
Conclusion
This process is obviously not suited for the automatic encoding process that a lot of sites seem to crave nowadays, this process is far better suited for the cash-strapped auteur who wants the most out of their videos and bandwidth and doesn't have a large amount of videos to encode. VP6 support in ffmpeg/libavcodec is coming along, and the most recent builds of ffmpeg come with decoding support for VP6, but whether patents/license prevent encoding support is still to be seen.