5 This script helps to convert video from one format to another.
6 This is useful for ripping DVD to mpeg4 video (XviD, DivX).
9 * automatic crop detection
10 * mp3 audio compression with resampling options
11 * automatic bitrate calculation based on desired target size
12 * optional interlace removal, b/w video optimization, video scaling
14 Run the script with no arguments to start with interactive prompts:
16 Run the script with the filename of a config to start automatic mode:
19 After Rippy is finished it saves the current configuation in a file called
20 'rippy.conf' in the local directoy. This can be used to rerun process using the
21 exact same settings by passing the filename of the conf file as an argument to
22 Rippy. Rippy will read the options from the file instead of asking you for
23 options interactively. So if you run rippy with 'dry_run=1' then you can run
24 the process again later using the 'rippy.conf' file. Don't forget to edit
25 'rippy.conf' to set 'dry_run=0'!
27 If you run rippy with 'dry_run' and 'verbose' true then the output generated is
28 valid command line commands. you could (in theory) cut-and-paste the commands
29 to a shell prompt. You will need to tweak some values such as crop area and bit
30 rate because these cannot be calculated in a dry run. This is useful if you
31 want to get an idea of what Rippy plans to do.
33 For all the trouble that Rippy goes through to calculate the best bitrate for a
34 desired target video size it sometimes fails to get it right. Sometimes the
35 final video size will differ more than you wanted from the desired size, but if
36 you are really motivated and have a lot of time on your hands then you can run
37 Rippy again with a manually calculated bitrate. After all compression is done
38 the first time Rippy will recalculate the bitrate to give you the nearly exact
39 bitrate that would have worked. You can then edit the 'rippy.conf' file; set
40 the video_bitrate with this revised bitrate; and then run Rippy all over again.
41 There is nothing like 4-pass video compression to get it right! Actually, this
42 could be done in three passes since I don't need to do the second pass
43 compression before I calculate the revised bitrate. I'm also considering an
44 enhancement where Rippy would compress ten spread out chunks, 1-minute in
45 length to estimate the bitrate.
47 Free, open source, and all that good stuff.
48 Rippy Copyright (c) 2006 Noah Spurrier
50 Permission is hereby granted, free of charge, to any person obtaining a copy
51 of this software and associated documentation files (the "Software"), to deal
52 in the Software without restriction, including without limitation the rights
53 to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
54 copies of the Software, and to permit persons to whom the Software is
55 furnished to do so, subject to the following conditions:
57 The above copyright notice and this permission notice shall be included in all
58 copies or substantial portions of the Software.
60 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
61 EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
62 MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
63 IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
64 DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
65 OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
66 USE OR OTHER DEALINGS IN THE SOFTWARE.
69 $Id: rippy.py 498 2007-12-17 13:44:19Z noah $
72 import sys, os, re, math, stat, getopt, traceback, types, time
76 __revision__ = '$Revision: 11 $'
77 __all__ = ['main', __version__, __revision__]
79 GLOBAL_LOGFILE_NAME = "rippy_%d.log" % os.getpid()
80 GLOBAL_LOGFILE = open (GLOBAL_LOGFILE_NAME, "wb")
82 ###############################################################################
83 # This giant section defines the prompts and defaults used in interactive mode.
84 ###############################################################################
85 # Python dictionaries are unordered, so
86 # I have this list that maintains the order of the keys.
90 'video_source_filename',
92 'video_final_filename',
96 'video_encode_passes',
98 'video_fourcc_override',
100 'video_bitrate_overhead',
103 'video_deinterlace_flag',
108 'audio_raw_filename',
109 'audio_volume_boost',
112 #'audio_lowpass_filter',
113 'delete_tmp_files_flag'
116 # The 'prompts' dictionary holds all the messages shown to the user in
117 # interactive mode. The 'prompts' dictionary schema is defined as follows:
118 # prompt_key : ( default value, prompt string, help string, level of difficulty (0,1,2) )
121 'video_source_filename':("dvd://1", 'video source filename?', """This is the filename of the video that you want to convert from.
122 It can be any file that mencoder supports.
123 You can also choose a DVD device using the dvd://1 syntax.
124 Title 1 is usually the main title on a DVD.""",0),
125 'video_chapter':("none",'video chapter?',"""This is the chapter number. Usually disks such as TV series seasons will be divided into chapters. Maybe be set to none.""",0),
126 'video_final_filename':("video_final.avi", "video final filename?", """This is the name of the final video.""",0),
127 'audio_raw_filename':("audiodump.wav", "audio raw filename?", """This is the audio raw PCM filename. This is prior to compression.
128 Note that mplayer automatically names this audiodump.wav, so don't change this.""",1000),
129 #'audio_compressed_filename':("audiodump.mp3","Audio compressed filename?", """This is the name of the compressed audio that will be mixed
130 #into the final video. Normally you don't need to change this.""",2),
131 'video_length':("none","video length in seconds?","""This sets the length of the video in seconds. This is used to estimate the
132 bitrate for a target video file size. Set to 'calc' to have Rippy calculate
133 the length. Set to 'none' if you don't want rippy to estimate the bitrate --
134 you will have to manually specify bitrate.""",1),
135 'video_aspect_ratio':("calc","aspect ratio?","""This sets the aspect ratio of the video. Most DVDs are 16/9 or 4/3.""",1),
136 'video_scale':("none","video scale?","""This scales the video to the given output size. The default is to do no scaling.
137 You may type in a resolution such as 320x240 or you may use presets.
138 qntsc: 352x240 (NTSC quarter screen)
139 qpal: 352x288 (PAL quarter screen)
140 ntsc: 720x480 (standard NTSC)
141 pal: 720x576 (standard PAL)
142 sntsc: 640x480 (square pixel NTSC)
143 spal: 768x576 (square pixel PAL)""",1),
144 'video_codec':("mpeg4","video codec?","""This is the video compression to use. This is passed directly to mencoder, so
145 any format that it recognizes should work. For XviD or DivX use mpeg4.
146 Almost all MS Windows systems support wmv2 out of the box.
147 Some common codecs include:
148 mjpeg, h263, h263p, h264, mpeg4, msmpeg4, wmv1, wmv2, mpeg1video, mpeg2video, huffyuv, ffv1.
150 'audio_codec':("mp3","audio codec?","""This is the audio compression to use. This is passed directly to mencoder, so
151 any format that it recognizes will work.
152 Some common codecs include:
154 See mencoder manual for details.""",2),
155 'video_fourcc_override':("XVID","force fourcc code?","""This forces the fourcc codec to the given value. XVID is safest for Windows.
156 The following are common fourcc values:
157 FMP4 - This is the mencoder default. This is the "real" value.
158 XVID - used by Xvid (safest)
160 MP4S - Microsoft""",2),
161 'video_encode_passes':("1","number of encode passes?","""This sets how many passes to use to encode the video. You can choose 1 or 2.
162 Using two pases takes twice as long as one pass, but produces a better
163 quality video. I found that the improvement is not that impressive.""",1),
164 'verbose_flag':("Y","verbose output?","""This sets verbose output. If true then all commands and arguments are printed
165 before they are run. This is useful to see exactly how commands are run.""",1),
166 'dry_run_flag':("N","dry run?","""This sets 'dry run' mode. If true then commands are not run. This is useful
167 if you want to see what would the script would do.""",1),
168 'video_bitrate':("calc","video bitrate?","""This sets the video bitrate. This overrides video_target_size.
169 Set to 'calc' to automatically estimate the bitrate based on the
170 video final target size. If you set video_length to 'none' then
171 you will have to specify this video_bitrate.""",1),
172 'video_target_size':("737280000","video final target size?","""This sets the target video size that you want to end up with.
173 This is over-ridden by video_bitrate. In other words, if you specify
174 video_bitrate then video_target_size is ignored.
175 Due to the unpredictable nature of VBR compression the final video size
176 may not exactly match. The following are common CDR sizes:
177 180MB CDR (21 minutes) holds 193536000 bytes
178 550MB CDR (63 minutes) holds 580608000 bytes
179 650MB CDR (74 minutes) holds 681984000 bytes
180 700MB CDR (80 minutes) holds 737280000 bytes""",0),
181 'video_bitrate_overhead':("1.0","bitrate overhead factor?","""Adjust this value if you want to leave more room for
182 other files such as subtitle files.
183 If you specify video_bitrate then this value is ignored.""",2),
184 'video_crop_area':("detect","crop area?","""This sets the crop area to remove black bars from the top or sides of the video.
185 This helps save space. Set to 'detect' to automatically detect the crop area.
186 Set to 'none' to not crop the video. Normally you don't need to change this.""",1),
187 'video_deinterlace_flag':("N","is the video interlaced?","""This sets the deinterlace flag. If set then mencoder will be instructed
188 to filter out interlace artifacts (using '-vf pp=md').""",1),
189 'video_gray_flag':("N","is the video black and white (gray)?","""This improves output for black and white video.""",1),
190 'subtitle_id':("None","Subtitle ID stream?","""This selects the subtitle stream to extract from the source video.
191 Normally, 0 is the English subtitle stream for a DVD.
192 Subtitles IDs with higher numbers may be other languages.""",1),
193 'audio_id':("128","audio ID stream?","""This selects the audio stream to extract from the source video.
194 If your source is a VOB file (DVD) then stream IDs start at 128.
195 Normally, 128 is the main audio track for a DVD.
196 Tracks with higher numbers may be other language dubs or audio commentary.""",1),
197 'audio_sample_rate':("32000","audio sample rate (Hz) 48000, 44100, 32000, 24000, 12000","""This sets the rate at which the compressed audio will be resampled.
198 DVD audio is 48 kHz whereas music CDs use 44.1 kHz. The higher the sample rate
199 the more space the audio track will take. That will leave less space for video.
200 32 kHz is a good trade-off if you are trying to fit a video onto a CD.""",1),
201 'audio_bitrate':("96","audio bitrate (kbit/s) 192, 128, 96, 64?","""This sets the bitrate for MP3 audio compression.
202 The higher the bitrate the more space the audio track will take.
203 That will leave less space for video. Most people find music to be acceptable
204 at 128 kBitS. 96 kBitS is a good trade-off if you are trying to fit a video onto a CD.""",1),
205 'audio_volume_boost':("none","volume dB boost?","""Many DVDs have very low audio volume. This sets an audio volume boost in Decibels.
206 Values of 6 to 10 usually adjust quiet DVDs to a comfortable level.""",1),
207 #'audio_lowpass_filter':("16","audio lowpass filter (kHz)?","""This sets the low-pass filter for the audio.
208 #Normally this should be half of the audio sample rate.
209 #This improves audio compression and quality.
210 #Normally you don't need to change this.""",1),
211 'delete_tmp_files_flag':("N","delete temporary files when finished?","""If Y then %s, audio_raw_filename, and 'divx2pass.log' will be deleted at the end."""%GLOBAL_LOGFILE_NAME,1)
214 ##############################################################################
215 # This is the important convert control function
216 ##############################################################################
217 def convert (options):
218 """This is the heart of it all -- this performs an end-to-end conversion of
219 a video from one format to another. It requires a dictionary of options.
220 The conversion process will also add some keys to the dictionary
221 such as length of the video and crop area. The dictionary is returned.
222 This options dictionary could be used again to repeat the convert process
223 (it is also saved to rippy.conf as text).
225 if options['subtitle_id'] is not None:
226 print "# extract subtitles"
227 apply_smart (extract_subtitles, options)
229 print "# do not extract subtitles."
232 # I really only need to calculate the exact video length if the user
233 # selected 'calc' for video_bitrate
235 # selected 'detect' for video_crop_area.
236 if options['video_bitrate']=='calc' or options['video_crop_area']=='detect':
237 # As strange as it seems, the only reliable way to calculate the length
238 # of a video (in seconds) is to extract the raw, uncompressed PCM audio stream
239 # and then calculate the length of that. This is because MP4 video is VBR, so
240 # you cannot get exact time based on compressed size.
241 if options['video_length']=='calc':
242 print "# extract PCM raw audio to %s" % (options['audio_raw_filename'])
243 apply_smart (extract_audio, options)
244 options['video_length'] = apply_smart (get_length, options)
245 print "# Length of raw audio file : %d seconds (%0.2f minutes)" % (options['video_length'], float(options['video_length'])/60.0)
246 if options['video_bitrate']=='calc':
247 options['video_bitrate'] = options['video_bitrate_overhead'] * apply_smart (calc_video_bitrate, options)
248 print "# video bitrate : " + str(options['video_bitrate'])
249 if options['video_crop_area']=='detect':
250 options['video_crop_area'] = apply_smart (crop_detect, options)
251 print "# crop area : " + str(options['video_crop_area'])
252 print "# compression estimate"
253 print apply_smart (compression_estimate, options)
255 print "# compress video"
256 apply_smart (compress_video, options)
257 'audio_volume_boost',
259 print "# delete temporary files:",
260 if options['delete_tmp_files_flag']:
262 apply_smart (delete_tmp_files, options)
266 # Finish by saving options to rippy.conf and
267 # calclating if final_size is less than target_size.
268 o = ["# options used to create video\n"]
269 video_actual_size = get_filesize (options['video_final_filename'])
270 if options['video_target_size'] != 'none':
271 revised_bitrate = calculate_revised_bitrate (options['video_bitrate'], options['video_target_size'], video_actual_size)
272 o.append("# revised video_bitrate : %d\n" % revised_bitrate)
273 for k,v in options.iteritems():
274 o.append (" %30s : %s\n" % (k, v))
276 fout = open("rippy.conf","wb").write(''.join(o))
277 print "# final actual video size = %d" % video_actual_size
278 if options['video_target_size'] != 'none':
279 if video_actual_size > options['video_target_size']:
280 print "# FINAL VIDEO SIZE IS GREATER THAN DESIRED TARGET"
281 print "# final video size is %d bytes over target size" % (video_actual_size - options['video_target_size'])
283 print "# final video size is %d bytes under target size" % (options['video_target_size'] - video_actual_size)
284 print "# If you want to run the entire compression process all over again"
285 print "# to get closer to the target video size then trying using a revised"
286 print "# video_bitrate of %d" % revised_bitrate
290 ##############################################################################
292 def exit_with_usage(exit_code=1):
293 print globals()['__doc__']
294 print 'version:', globals()['__version__']
298 def check_missing_requirements ():
299 """This list of missing requirements (mencoder, mplayer, lame, and mkvmerge).
300 Returns None if all requirements are in the execution path.
303 if pexpect.which("mencoder") is None:
304 missing.append("mencoder")
305 if pexpect.which("mplayer") is None:
306 missing.append("mplayer")
307 #if pexpect.which("lame") is None:
308 # missing.append("lame")
309 #if pexpect.which("mkvmerge") is None:
310 # missing.append("mkvmerge")
315 def input_option (message, default_value="", help=None, level=0, max_level=0):
316 """This is a fancy raw_input function.
317 If the user enters '?' then the contents of help is printed.
319 The 'level' and 'max_level' are used to adjust which advanced options
320 are printed. 'max_level' is the level of options that the user wants
321 to see. 'level' is the level of difficulty for this particular option.
322 If this level is <= the max_level the user wants then the
323 message is printed and user input is allowed; otherwise, the
324 default value is returned automatically without user input.
326 if default_value != '':
327 message = "%s [%s] " % (message, default_value)
328 if level > max_level:
331 user_input = raw_input (message)
340 def progress_callback (d=None):
341 """This callback simply prints a dot to show activity.
342 This is used when running external commands with pexpect.run.
344 sys.stdout.write (".")
348 global GLOBAL_LOGFILE
349 print >>GLOBAL_LOGFILE, cmd
350 (command_output, exitstatus) = pexpect.run(cmd, events={pexpect.TIMEOUT:progress_callback}, timeout=5, withexitstatus=True, logfile=GLOBAL_LOGFILE)
352 print "RUN FAILED. RETURNED EXIT STATUS:", exitstatus
353 print >>GLOBAL_LOGFILE, "RUN FAILED. RETURNED EXIT STATUS:", exitstatus
354 return (command_output, exitstatus)
356 def apply_smart (func, args):
357 """This is similar to func(**args), but this won't complain about
358 extra keys in 'args'. This ignores keys in 'args' that are
359 not required by 'func'. This passes None to arguments that are
360 not defined in 'args'. That's fine for arguments with a default valeue, but
361 that's a bug for required arguments. I should probably raise a TypeError.
362 The func parameter can be a function reference or a string.
363 If it is a string then it is converted to a function reference.
365 if type(func) is type(''):
366 if func in globals():
367 func = globals()[func]
369 raise NameError("name '%s' is not defined" % func)
370 if hasattr(func,'im_func'): # Handle case when func is a class method.
372 argcount = func.func_code.co_argcount
373 required_args = dict([(k,args.get(k)) for k in func.func_code.co_varnames[:argcount]])
374 return func(**required_args)
376 def count_unique (items):
377 """This takes a list and returns a sorted list of tuples with a count of each unique item in the list.
379 count_unique(['a','b','c','a','c','c','a','c','c'])
381 [(5,'c'), (3,'a'), (1,'b')]
382 Example 2 -- get the most frequent item in a list:
383 count_unique(['a','b','c','a','c','c','a','c','c'])[0][1]
390 stats[i] = stats[i] + 1
393 stats = [(v, k) for k, v in stats.items()]
398 def calculate_revised_bitrate (video_bitrate, video_target_size, video_actual_size):
399 """This calculates a revised video bitrate given the video_bitrate used,
400 the actual size that resulted, and the video_target_size.
401 This can be used if you want to compress the video all over again in an
402 attempt to get closer to the video_target_size.
404 return int(math.floor(video_bitrate * (float(video_target_size) / float(video_actual_size))))
406 def get_aspect_ratio (video_source_filename):
407 """This returns the aspect ratio of the original video.
408 This is usualy 1.78:1(16/9) or 1.33:1(4/3).
409 This function is very lenient. It basically guesses 16/9 whenever
410 it cannot figure out the aspect ratio.
412 cmd = "mplayer '%s' -vo png -ao null -frames 1" % video_source_filename
413 (command_output, exitstatus) = run(cmd)
414 ar = re.findall("Movie-Aspect is ([0-9]+\.?[0-9]*:[0-9]+\.?[0-9]*)", command_output)
417 if ar[0] == '1.78:1':
419 if ar[0] == '1.33:1':
422 #idh = re.findall("ID_VIDEO_HEIGHT=([0-9]+)", command_output)
423 #if len(idw)==0 or len(idh)==0:
425 # print 'Could not get aspect ration. Assuming 1.78:1 (16/9).'
427 #return float(idw[0])/float(idh[0])
430 #Movie-Aspect is 1.78:1 - prescaling to correct movie aspect.
433 def get_aid_list (video_source_filename):
434 """This returns a list of audio ids in the source video file.
435 TODO: Also extract ID_AID_nnn_LANG to associate language. Not all DVDs include this.
437 cmd = "mplayer '%s' -vo null -ao null -frames 0 -identify" % video_source_filename
438 (command_output, exitstatus) = run(cmd)
439 idl = re.findall("ID_AUDIO_ID=([0-9]+)", command_output)
443 def get_sid_list (video_source_filename):
444 """This returns a list of subtitle ids in the source video file.
445 TODO: Also extract ID_SID_nnn_LANG to associate language. Not all DVDs include this.
447 cmd = "mplayer '%s' -vo null -ao null -frames 0 -identify" % video_source_filename
448 (command_output, exitstatus) = run(cmd)
449 idl = re.findall("ID_SUBTITLE_ID=([0-9]+)", command_output)
453 def extract_audio (video_source_filename, audio_id=128, verbose_flag=0, dry_run_flag=0):
454 """This extracts the given audio_id track as raw uncompressed PCM from the given source video.
455 Note that mplayer always saves this to audiodump.wav.
456 At this time there is no way to set the output audio name.
458 #cmd = "mplayer %(video_source_filename)s -vc null -vo null -aid %(audio_id)s -ao pcm:fast -noframedrop" % locals()
459 cmd = "mplayer -quiet '%(video_source_filename)s' -vc dummy -vo null -aid %(audio_id)s -ao pcm:fast -noframedrop" % locals()
460 if verbose_flag: print cmd
465 def extract_subtitles (video_source_filename, subtitle_id=0, verbose_flag=0, dry_run_flag=0):
466 """This extracts the given subtitle_id track as VOBSUB format from the given source video.
468 cmd = "mencoder -quiet '%(video_source_filename)s' -o /dev/null -nosound -ovc copy -vobsubout subtitles -vobsuboutindex 0 -sid %(subtitle_id)s" % locals()
469 if verbose_flag: print cmd
474 def get_length (audio_raw_filename):
475 """This attempts to get the length of the media file (length is time in seconds).
476 This should not be confused with size (in bytes) of the file data.
477 This is best used on a raw PCM AUDIO file because mplayer cannot get an accurate
478 time for many compressed video and audio formats -- notably MPEG4 and MP3.
480 This returns -1 if it cannot get the length of the given file.
482 cmd = "mplayer %s -vo null -ao null -frames 0 -identify" % audio_raw_filename
483 (command_output, exitstatus) = run(cmd)
484 idl = re.findall("ID_LENGTH=([0-9.]*)", command_output)
487 print "ERROR: cannot get length of raw audio file."
488 print "command_output of mplayer identify:"
490 print "parsed command_output:"
495 def get_filesize (filename):
496 """This returns the number of bytes a file takes on storage."""
497 return os.stat(filename)[stat.ST_SIZE]
499 def calc_video_bitrate (video_target_size, audio_bitrate, video_length, extra_space=0, dry_run_flag=0):
500 """This gives an estimate of the video bitrate necessary to
501 fit the final target size. This will take into account room to
502 fit the audio and extra space if given (for container overhead or whatnot).
503 video_target_size is in bytes,
504 audio_bitrate is bits per second (96, 128, 256, etc.) ASSUMING CBR,
505 video_length is in seconds,
506 extra_space is in bytes.
507 a 180MB CDR (21 minutes) holds 193536000 bytes.
508 a 550MB CDR (63 minutes) holds 580608000 bytes.
509 a 650MB CDR (74 minutes) holds 681984000 bytes.
510 a 700MB CDR (80 minutes) holds 737280000 bytes.
514 if extra_space is None: extra_space = 0
515 #audio_size = os.stat(audio_compressed_filename)[stat.ST_SIZE]
516 audio_size = (audio_bitrate * video_length * 1000) / 8.0
517 video_target_size = video_target_size - audio_size - extra_space
518 return (int)(calc_video_kbitrate (video_target_size, video_length))
520 def calc_video_kbitrate (target_size, length_secs):
521 """Given a target byte size free for video data, this returns the bitrate in kBit/S.
522 For mencoder vbitrate 1 kBit = 1000 Bits -- not 1024 bits.
523 target_size = bitrate * 1000 * length_secs / 8
524 target_size = bitrate * 125 * length_secs
525 bitrate = target_size/(125*length_secs)
527 return int(target_size / (125.0 * length_secs))
529 def crop_detect (video_source_filename, video_length, dry_run_flag=0):
530 """This attempts to figure out the best crop for the given video file.
531 Basically it runs crop detect for 10 seconds on five different places in the video.
532 It picks the crop area that was most often detected.
534 skip = int(video_length/9) # offset to skip (-ss option in mencoder)
536 cmd1 = "mencoder '%s' -quiet -ss %d -endpos %d -o /dev/null -nosound -ovc lavc -vf cropdetect" % (video_source_filename, skip, sample_length)
537 cmd2 = "mencoder '%s' -quiet -ss %d -endpos %d -o /dev/null -nosound -ovc lavc -vf cropdetect" % (video_source_filename, 2*skip, sample_length)
538 cmd3 = "mencoder '%s' -quiet -ss %d -endpos %d -o /dev/null -nosound -ovc lavc -vf cropdetect" % (video_source_filename, 4*skip, sample_length)
539 cmd4 = "mencoder '%s' -quiet -ss %d -endpos %d -o /dev/null -nosound -ovc lavc -vf cropdetect" % (video_source_filename, 6*skip, sample_length)
540 cmd5 = "mencoder '%s' -quiet -ss %d -endpos %d -o /dev/null -nosound -ovc lavc -vf cropdetect" % (video_source_filename, 8*skip, sample_length)
543 (command_output1, exitstatus1) = run(cmd1)
544 (command_output2, exitstatus2) = run(cmd2)
545 (command_output3, exitstatus3) = run(cmd3)
546 (command_output4, exitstatus4) = run(cmd4)
547 (command_output5, exitstatus5) = run(cmd5)
548 idl = re.findall("-vf crop=([0-9]+:[0-9]+:[0-9]+:[0-9]+)", command_output1)
549 idl = idl + re.findall("-vf crop=([0-9]+:[0-9]+:[0-9]+:[0-9]+)", command_output2)
550 idl = idl + re.findall("-vf crop=([0-9]+:[0-9]+:[0-9]+:[0-9]+)", command_output3)
551 idl = idl + re.findall("-vf crop=([0-9]+:[0-9]+:[0-9]+:[0-9]+)", command_output4)
552 idl = idl + re.findall("-vf crop=([0-9]+:[0-9]+:[0-9]+:[0-9]+)", command_output5)
553 items_count = count_unique(idl)
554 return items_count[0][1]
557 def build_compression_command (video_source_filename, video_final_filename, video_target_size, audio_id=128, video_bitrate=1000, video_codec='mpeg4', audio_codec='mp3', video_fourcc_override='FMP4', video_gray_flag=0, video_crop_area=None, video_aspect_ratio='16/9', video_scale=None, video_encode_passes=2, video_deinterlace_flag=0, audio_volume_boost=None, audio_sample_rate=None, audio_bitrate=None, seek_skip=None, seek_length=None, video_chapter=None):
558 #Notes:For DVD, VCD, and SVCD use acodec=mp2 and vcodec=mpeg2video:
559 #mencoder movie.avi -o movie.VOB -ovc lavc -oac lavc -lavcopts acodec=mp2:abitrate=224:vcodec=mpeg2video:vbitrate=2000
562 # build video filter (-vf) argument
565 if video_crop_area and video_crop_area.lower()!='none':
566 video_filter = video_filter + 'crop=%s' % video_crop_area
567 if video_deinterlace_flag:
568 if video_filter != '':
569 video_filter = video_filter + ','
570 video_filter = video_filter + 'pp=md'
571 if video_scale and video_scale.lower()!='none':
572 if video_filter != '':
573 video_filter = video_filter + ','
574 video_filter = video_filter + 'scale=%s' % video_scale
575 # optional video rotation -- were you holding your camera sideways?
576 #if video_filter != '':
577 # video_filter = video_filter + ','
578 #video_filter = video_filter + 'rotate=2'
579 if video_filter != '':
580 video_filter = '-vf ' + video_filter
583 # build chapter argument
585 if video_chapter is not None:
586 chapter = '-chapter %d-%d' %(video_chapter,video_chapter)
589 # chapter = '-chapter 2-2'
592 # build audio_filter argument
595 if audio_sample_rate:
596 if audio_filter != '':
597 audio_filter = audio_filter + ','
598 audio_filter = audio_filter + 'lavcresample=%s' % audio_sample_rate
599 if audio_volume_boost is not None:
600 if audio_filter != '':
601 audio_filter = audio_filter + ','
602 audio_filter = audio_filter + 'volume=%0.1f:1'%audio_volume_boost
603 if audio_filter != '':
604 audio_filter = '-af ' + audio_filter
606 #if audio_sample_rate:
607 # audio_filter = ('-srate %d ' % audio_sample_rate) + audio_filter
610 # build lavcopts argument
612 #lavcopts = '-lavcopts vcodec=%s:vbitrate=%d:mbd=2:aspect=%s:acodec=%s:abitrate=%d:vpass=1' % (video_codec,video_bitrate,audio_codec,audio_bitrate)
613 lavcopts = '-lavcopts vcodec=%(video_codec)s:vbitrate=%(video_bitrate)d:mbd=2:aspect=%(video_aspect_ratio)s:acodec=%(audio_codec)s:abitrate=%(audio_bitrate)d:vpass=1' % (locals())
615 lavcopts = lavcopts + ':gray'
618 if seek_skip is not None:
619 seek_filter = '-ss %s' % (str(seek_skip))
620 if seek_length is not None:
621 seek_filter = seek_filter + ' -endpos %s' % (str(seek_length))
623 cmd = "mencoder -quiet -info comment='Arkivist' '%(video_source_filename)s' %(seek_filter)s %(chapter)s -aid %(audio_id)s -o '%(video_final_filename)s' -ffourcc %(video_fourcc_override)s -ovc lavc -oac lavc %(lavcopts)s %(video_filter)s %(audio_filter)s" % locals()
626 def compression_estimate (video_length, video_source_filename, video_final_filename, video_target_size, audio_id=128, video_bitrate=1000, video_codec='mpeg4', audio_codec='mp3', video_fourcc_override='FMP4', video_gray_flag=0, video_crop_area=None, video_aspect_ratio='16/9', video_scale=None, video_encode_passes=2, video_deinterlace_flag=0, audio_volume_boost=None, audio_sample_rate=None, audio_bitrate=None):
627 """This attempts to figure out the best compression ratio for a given set of compression options.
629 # TODO Need to account for AVI overhead.
630 skip = int(video_length/9) # offset to skip (-ss option in mencoder)
632 cmd1 = build_compression_command (video_source_filename, "compression_test_1.avi", video_target_size, audio_id, video_bitrate, video_codec, audio_codec, video_fourcc_override, video_gray_flag, video_crop_area, video_aspect_ratio, video_scale, video_encode_passes, video_deinterlace_flag, audio_volume_boost, audio_sample_rate, audio_bitrate, skip, sample_length)
633 cmd2 = build_compression_command (video_source_filename, "compression_test_2.avi", video_target_size, audio_id, video_bitrate, video_codec, audio_codec, video_fourcc_override, video_gray_flag, video_crop_area, video_aspect_ratio, video_scale, video_encode_passes, video_deinterlace_flag, audio_volume_boost, audio_sample_rate, audio_bitrate, skip*2, sample_length)
634 cmd3 = build_compression_command (video_source_filename, "compression_test_3.avi", video_target_size, audio_id, video_bitrate, video_codec, audio_codec, video_fourcc_override, video_gray_flag, video_crop_area, video_aspect_ratio, video_scale, video_encode_passes, video_deinterlace_flag, audio_volume_boost, audio_sample_rate, audio_bitrate, skip*4, sample_length)
635 cmd4 = build_compression_command (video_source_filename, "compression_test_4.avi", video_target_size, audio_id, video_bitrate, video_codec, audio_codec, video_fourcc_override, video_gray_flag, video_crop_area, video_aspect_ratio, video_scale, video_encode_passes, video_deinterlace_flag, audio_volume_boost, audio_sample_rate, audio_bitrate, skip*6, sample_length)
636 cmd5 = build_compression_command (video_source_filename, "compression_test_5.avi", video_target_size, audio_id, video_bitrate, video_codec, audio_codec, video_fourcc_override, video_gray_flag, video_crop_area, video_aspect_ratio, video_scale, video_encode_passes, video_deinterlace_flag, audio_volume_boost, audio_sample_rate, audio_bitrate, skip*8, sample_length)
642 size = get_filesize ("compression_test_1.avi")+get_filesize ("compression_test_2.avi")+get_filesize ("compression_test_3.avi")+get_filesize ("compression_test_4.avi")+get_filesize ("compression_test_5.avi")
645 def compress_video (video_source_filename, video_final_filename, video_target_size, audio_id=128, video_bitrate=1000, video_codec='mpeg4', audio_codec='mp3', video_fourcc_override='FMP4', video_gray_flag=0, video_crop_area=None, video_aspect_ratio='16/9', video_scale=None, video_encode_passes=2, video_deinterlace_flag=0, audio_volume_boost=None, audio_sample_rate=None, audio_bitrate=None, seek_skip=None, seek_length=None, video_chapter=None, verbose_flag=0, dry_run_flag=0):
646 """This compresses the video and audio of the given source video filename to the transcoded filename.
647 This does a two-pass compression (I'm assuming mpeg4, I should probably make this smarter for other formats).
650 # do the first pass video compression
652 #cmd = "mencoder -quiet '%(video_source_filename)s' -ss 65 -endpos 20 -aid %(audio_id)s -o '%(video_final_filename)s' -ffourcc %(video_fourcc_override)s -ovc lavc -oac lavc %(lavcopts)s %(video_filter)s %(audio_filter)s" % locals()
654 cmd = build_compression_command (video_source_filename, video_final_filename, video_target_size, audio_id, video_bitrate, video_codec, audio_codec, video_fourcc_override, video_gray_flag, video_crop_area, video_aspect_ratio, video_scale, video_encode_passes, video_deinterlace_flag, audio_volume_boost, audio_sample_rate, audio_bitrate, seek_skip, seek_length, video_chapter)
655 if verbose_flag: print cmd
660 # If not doing two passes then return early.
661 if video_encode_passes!='2':
665 video_actual_size = get_filesize (video_final_filename)
666 if video_actual_size > video_target_size:
667 print "======================================================="
669 print "First pass compression resulted in"
670 print "actual file size greater than target size."
671 print "Second pass will be too big."
672 print "======================================================="
675 # do the second pass video compression
677 cmd = cmd.replace ('vpass=1', 'vpass=2')
678 if verbose_flag: print cmd
684 def compress_audio (audio_raw_filename, audio_compressed_filename, audio_lowpass_filter=None, audio_sample_rate=None, audio_bitrate=None, verbose_flag=0, dry_run_flag=0):
685 """This is depricated.
686 This compresses the raw audio file to the compressed audio filename.
688 cmd = 'lame -h --athaa-sensitivity 1' # --cwlimit 11"
689 if audio_lowpass_filter:
690 cmd = cmd + ' --lowpass ' + audio_lowpass_filter
692 #cmd = cmd + ' --abr ' + audio_bitrate
693 cmd = cmd + ' --cbr -b ' + audio_bitrate
694 if audio_sample_rate:
695 cmd = cmd + ' --resample ' + audio_sample_rate
696 cmd = cmd + ' ' + audio_raw_filename + ' ' + audio_compressed_filename
697 if verbose_flag: print cmd
699 (command_output, exitstatus) = run(cmd)
702 raise Exception('ERROR: lame failed to compress raw audio file.')
704 def mux (video_final_filename, video_transcoded_filename, audio_compressed_filename, video_container_format, verbose_flag=0, dry_run_flag=0):
705 """This is depricated. I used to use a three-pass encoding where I would mix the audio track separately, but
706 this never worked very well (loss of audio sync)."""
707 if video_container_format.lower() == 'mkv': # Matroska
708 mux_mkv (video_final_filename, video_transcoded_filename, audio_compressed_filename, verbose_flag, dry_run_flag)
709 if video_container_format.lower() == 'avi':
710 mux_avi (video_final_filename, video_transcoded_filename, audio_compressed_filename, verbose_flag, dry_run_flag)
712 def mux_mkv (video_final_filename, video_transcoded_filename, audio_compressed_filename, verbose_flag=0, dry_run_flag=0):
713 """This is depricated."""
714 cmd = 'mkvmerge -o %s --noaudio %s %s' % (video_final_filename, video_transcoded_filename, audio_compressed_filename)
715 if verbose_flag: print cmd
720 def mux_avi (video_final_filename, video_transcoded_filename, audio_compressed_filename, verbose_flag=0, dry_run_flag=0):
721 """This is depricated."""
722 cmd = "mencoder -quiet -oac copy -ovc copy -o '%s' -audiofile %s '%s'" % (video_final_filename, audio_compressed_filename, video_transcoded_filename)
723 if verbose_flag: print cmd
728 def delete_tmp_files (audio_raw_filename, verbose_flag=0, dry_run_flag=0):
729 global GLOBAL_LOGFILE_NAME
730 file_list = ' '.join([GLOBAL_LOGFILE_NAME, 'divx2pass.log', audio_raw_filename ])
731 cmd = 'rm -f ' + file_list
732 if verbose_flag: print cmd
737 ##############################################################################
738 # This is the interactive Q&A that is used if a conf file was not given.
739 ##############################################################################
740 def interactive_convert ():
741 global prompts, prompts_key_order
743 print globals()['__doc__']
745 print "=============================================="
746 print " Enter '?' at any question to get extra help."
747 print "=============================================="
750 # Ask for the level of options the user wants.
751 # A lot of code just to print a string!
752 level_sort = {0:'', 1:'', 2:''}
754 level = prompts[k][3]
755 if level < 0 or level > 2:
757 level_sort[level] += " " + prompts[k][1] + "\n"
758 level_sort_string = "This sets the level for advanced options prompts. Set 0 for simple, 1 for advanced, or 2 for expert.\n"
759 level_sort_string += "[0] Basic options:\n" + str(level_sort[0]) + "\n"
760 level_sort_string += "[1] Advanced options:\n" + str(level_sort[1]) + "\n"
761 level_sort_string += "[2] Expert options:\n" + str(level_sort[2])
762 c = input_option("Prompt level (0, 1, or 2)?", "1", level_sort_string)
763 max_prompt_level = int(c)
766 for k in prompts_key_order:
767 if k == 'video_aspect_ratio':
768 guess_aspect = get_aspect_ratio(options['video_source_filename'])
769 options[k] = input_option (prompts[k][1], guess_aspect, prompts[k][2], prompts[k][3], max_prompt_level)
770 elif k == 'audio_id':
771 aid_list = get_aid_list (options['video_source_filename'])
773 if max_prompt_level>=prompts[k][3]:
774 if len(aid_list) > 1:
775 print "This video has more than one audio stream. The following stream audio IDs were found:"
778 default_id = aid_list[0]
781 print "Rippy was unable to get the list of audio streams from this video."
782 print "If reading directly from a DVD then the DVD device might be busy."
783 print "Using a default setting of stream id 128 (main audio on most DVDs)."
785 options[k] = input_option (prompts[k][1], default_id, prompts[k][2], prompts[k][3], max_prompt_level)
786 elif k == 'subtitle_id':
787 sid_list = get_sid_list (options['video_source_filename'])
789 if max_prompt_level>=prompts[k][3]:
790 if len(sid_list) > 0:
791 print "This video has one or more subtitle streams. The following stream subtitle IDs were found:"
794 #default_id = sid_list[0]
795 default_id = prompts[k][0]
798 print "Unable to get the list of subtitle streams from this video. It may have none."
799 print "Setting default to None."
801 options[k] = input_option (prompts[k][1], default_id, prompts[k][2], prompts[k][3], max_prompt_level)
802 elif k == 'audio_lowpass_filter':
803 lowpass_default = "%.1f" % (math.floor(float(options['audio_sample_rate']) / 2.0))
804 options[k] = input_option (prompts[k][1], lowpass_default, prompts[k][2], prompts[k][3], max_prompt_level)
805 elif k == 'video_bitrate':
806 if options['video_length'].lower() == 'none':
807 options[k] = input_option (prompts[k][1], '1000', prompts[k][2], prompts[k][3], max_prompt_level)
809 options[k] = input_option (prompts[k][1], prompts[k][0], prompts[k][2], prompts[k][3], max_prompt_level)
811 # don't bother asking for video_target_size or video_bitrate_overhead if video_bitrate was set
812 if (k=='video_target_size' or k=='video_bitrate_overhead') and options['video_bitrate']!='calc':
814 # don't bother with crop area if video length is none
815 if k == 'video_crop_area' and options['video_length'].lower() == 'none':
816 options['video_crop_area'] = 'none'
818 options[k] = input_option (prompts[k][1], prompts[k][0], prompts[k][2], prompts[k][3], max_prompt_level)
820 #options['video_final_filename'] = options['video_final_filename'] + "." + options['video_container_format']
822 print "=========================================================================="
823 print "Ready to Rippy!"
825 print "The following options will be used:"
826 for k,v in options.iteritems():
827 print "%27s : %s" % (k, v)
830 c = input_option("Continue?", "Y")
831 c = c.strip().lower()
837 def clean_options (d):
838 """This validates and cleans up the options dictionary.
839 After reading options interactively or from a conf file
840 we need to make sure that the values make sense and are
841 converted to the correct type.
842 1. Any key with "_flag" in it becomes a boolean True or False.
843 2. Values are normalized ("No", "None", "none" all become "none";
844 "Calcluate", "c", "CALC" all become "calc").
845 3. Certain values are converted from string to int.
846 4. Certain combinations of options are invalid or override each other.
847 This is a rather annoying function, but then so it most cleanup work.
851 # convert all flag options to 0 or 1
853 if type(d[k]) is types.StringType:
854 if d[k].strip().lower()[0] in 'yt1': #Yes, True, 1
858 d['video_bitrate'] = d['video_bitrate'].lower()
859 if d['video_bitrate'][0]=='c':
860 d['video_bitrate']='calc'
862 d['video_bitrate'] = int(float(d['video_bitrate']))
864 d['video_target_size'] = int(d['video_target_size'])
865 # shorthand magic numbers get automatically expanded
866 if d['video_target_size'] == 180:
867 d['video_target_size'] = 193536000
868 elif d['video_target_size'] == 550:
869 d['video_target_size'] = 580608000
870 elif d['video_target_size'] == 650:
871 d['video_target_size'] = 681984000
872 elif d['video_target_size'] == 700:
873 d['video_target_size'] = 737280000
875 d['video_target_size'] = 'none'
878 d['video_chapter'] = int(d['video_chapter'])
880 d['video_chapter'] = None
883 d['subtitle_id'] = int(d['subtitle_id'])
885 d['subtitle_id'] = None
888 d['video_bitrate_overhead'] = float(d['video_bitrate_overhead'])
890 d['video_bitrate_overhead'] = -1.0
892 d['audio_bitrate'] = int(d['audio_bitrate'])
893 d['audio_sample_rate'] = int(d['audio_sample_rate'])
894 d['audio_volume_boost'] = d['audio_volume_boost'].lower()
895 if d['audio_volume_boost'][0]=='n':
896 d['audio_volume_boost'] = None
898 d['audio_volume_boost'] = d['audio_volume_boost'].replace('db','')
899 d['audio_volume_boost'] = float(d['audio_volume_boost'])
901 # assert (d['video_bitrate']=='calc' and d['video_target_size']!='none')
902 # or (d['video_bitrate']!='calc' and d['video_target_size']=='none')
904 d['video_scale'] = d['video_scale'].lower()
905 if d['video_scale'][0]=='n':
906 d['video_scale']='none'
908 al = re.findall("([0-9]+).*?([0-9]+)", d['video_scale'])
909 d['video_scale']=al[0][0]+':'+al[0][1]
910 d['video_crop_area'] = d['video_crop_area'].lower()
911 if d['video_crop_area'][0]=='n':
912 d['video_crop_area']='none'
913 d['video_length'] = d['video_length'].lower()
914 if d['video_length'][0]=='c':
915 d['video_length']='calc'
916 elif d['video_length'][0]=='n':
917 d['video_length']='none'
919 d['video_length'] = int(float(d['video_length']))
920 if d['video_length']==0:
921 d['video_length'] = 'none'
922 assert (not (d['video_length']=='none' and d['video_bitrate']=='calc'))
927 optlist, args = getopt.getopt(sys.argv[1:], 'h?', ['help','h','?'])
931 command_line_options = dict(optlist)
932 # There are a million ways to cry for help. These are but a few of them.
933 if [elem for elem in command_line_options if elem in ['-h','--h','-?','--?','--help']]:
936 missing = check_missing_requirements()
937 if missing is not None:
939 print "=========================================================================="
941 print "Some required external commands are missing."
942 print "please install the following packages:"
944 print "=========================================================================="
946 c = input_option("Continue?", "Y")
947 c = c.strip().lower()
953 # cute one-line string-to-dictionary parser (two-lines if you count this comment):
954 options = dict(re.findall('([^: \t\n]*)\s*:\s*(".*"|[^ \t\n]*)', file(args[0]).read()))
955 options = clean_options(options)
958 options = interactive_convert ()
959 options = clean_options(options)
963 if __name__ == "__main__":
965 start_time = time.time()
969 print "TOTAL TIME IN MINUTES:",
970 print (time.time() - start_time) / 60.0
972 tb_dump = traceback.format_exc()
973 print "=========================================================================="
974 print "ERROR -- Unexpected exception in script."
977 print "=========================================================================="
978 print >>GLOBAL_LOGFILE, "=========================================================================="
979 print >>GLOBAL_LOGFILE, "ERROR -- Unexpected exception in script."
980 print >>GLOBAL_LOGFILE, str(e)
981 print >>GLOBAL_LOGFILE, str(tb_dump)
982 print >>GLOBAL_LOGFILE, "=========================================================================="