November 28, 2007

iTunes 7.5 Watched Folders / Folder Sync (That Doesn't Suck)

As a long time WinAmp user and iPod owner, one of the many things that annoys me is its inability to sync to a folder or folders. I only use iTunes for managing my iPods, and a number of other programs for working with my video and music collection. I've tried various utilities in the past, most require a 50mb .NET Framework download and seem to break with every new iTunes version. So after hearing about the iTunes SDK, and that you could work with it via Windows Scripting Host in JavaScript (a language I know very well), I decided to write my own. The nice thing about using the COM interface is you've telling iTunes to work with its database on your behalf, so the script won't break when iTunes 8 is released for example. Oh and my script is only 12KB.

The goals for the my sync script were to: Add any new files that weren't in the iTunes database, remove any that were missing* and check if the MP4 or iD3 tags in the files have been updated (with Tag&Rename or Atomic Parsley for example) and make the necessary updates in iTunes. Some of my media is on external drives, video especially, so if the base folder for a certain media type is offline the script doesn't remove any missing media files of that type. When the drive with these files is reconnected everything will still work correctly in iTunes.

Currently the script has base folders for Music, Films and TV Shows (music videos are treated as normal music files). The script looks for files directly inside these folders and in subfolders one level deep. It checks files of extensions .mp3, .mp4, .m4a and .m4v. If you need something other than this setup, feel free to modify the script as necessary, but please don't ask me to make changes as I really don't have the time and this is how my library is setup. The code is well commented enough to be a good starting point for making your own script though.

When you run it a dialog will appear with your current base folders (these are stored in HKCU in the registry). Clicking YES will proceed with the folders displayed, NO will prompt you for folders and then proceed, and CANCEL will exit without doing anything. The script will start iTunes if it isn't running and start checking, there's no progress display (sorry I don't think this is possible with WSH) and will take about 30 seconds on most libraries (mine has 8000 items and averages about that), it will display a dialog with changes made when complete. If it finds a large number of items to add, it will prompt you with a dialog prior to this with a time estimate and ask whether you wish to proceed with the add operation. The script only modifies the iTunes database, it makes no changes to the actual media files themselves.

Notes: For files were the tags have been updated outside iTunes, things like Play Counts etc are preserved. You can also just press ENTER on the main dialog to proceed.

iTunesFolderSync.wsf (12KB)

Great hall locations: iPod  |  Talked about by: 0 Fraggles

July 31, 2007

LAME + EAC Settings

UPDATED 2008-03-07: Removed genre from tagging since the one supplied by FreeDB is usually wrong anyway.

These are the LAME settings I'm using in EAC, configured to use LAME as a custom encoder. See this Hydrogenaudio Knowledgebase page for further info.

-V 0 --vbr-new --id3v2-only --pad-id3v2 --ta "%a" --tt "%t" --tl "%g" --ty "%y" --tn "%n" --ignore-tag-errors %s %d

Great hall locations: iPod  |  Talked about by: 0 Fraggles

July 4, 2007

eMusic & Spam

Interesting this... when I signed up to eMusic I created an unique email address on one of my domains for the account. The only place I used this address was on the eMusic.com sign up form. Now I'm starting to receive spam on that address, and only on that address. After checking on the net, it appears they share their user database with anyone and everyone. Thank goodness I can just drop the address...

UPDATE: Oh and I just got another spam e-mail, and the To address just happened to contain the 2 unique addresses I had used to sign up to Emusic with (I cancelled and tried it again about 18 months later). Coincidence I think not!!

Great hall locations: iPod  |  Talked about by: 4 Fraggles

March 9, 2007

iTunes Library Updater

This utility solves one my biggest annoyances with iTunes, it can't sync its library to a folder on my computer (like WinAmp does). I use external tools for tagging/organising my music, and hence things get renamed, added and moved around outside of iTunes causing it to lose the plot—not anymore ;-)

Great hall locations: iPod  |  Talked about by: 0 Fraggles

August 30, 2006

Step By Step Conversion Guide: Putting DVD's On To Your iPod (revised)

UPDATED: 22nd September 2007.
(MP4 Scripts updated September 17th 2007)

These settings/scripts will generate files of pretty much transparent quality to the original DVD. I'm using these settings for archival purposes, so if you're looking for really small files, these might not be for you.

While I have previously written about this topic, I never covered it in any depth or went into every step I follow. When I first got my G5 iPod it was a steep learning curve figuring out MPEG video; what tools to use, what settings to use, what version of each software app was best, etc. to get the best possible quality and file size.

This post should serve as a complete answer to the question of "How do I get DVD's onto my iPod that look good on both the iPod Screen, a TV at the smallest possible file size". Below is the step by step process complete with illustrations and links to the required software.

DISCLAIMER

THIS GUIDE IS ONLY FOR CONVERTING MOVIES TO VIEW ON YOUR IPOD THAT YOU LEGALLY OWN, I.E. YOU PAID FOR THE DVD.

WE DO NOT SUPPORT THE PIRATING OF MOVIES, AND ANY COMMENTS OR MESSAGES RELATING TO THIS WILL BE DELETED. UNCLE TRAVELING MATT.COM ACCEPTS NO RESPONSIBILITY FOR THE USE OR MISUSE OF ANY OF THE SOFTWARE MENTIONED ON THIS SITE. UNCLE TRAVELING MATT.COM DOES NOT HOST AND DID NOT WRITE ANY OF THE SOFTWARE LINKED TO HERE, EXCEPT FOR THE BATCH FILE SCRIPTS THAT INTERACT WITH THE AFOREMENTIONED SOFTWARE. SO IF THESE MESS UP YOUR COMPUTER OR RUN OFF WITH YOUR DOG IT'S NOT OUR PROBLEM, BY USING THIS INFORMATION YOU ARE AGREEING TO THESE TERMS.

Required Software

* You are welcome to try newer versions than those listed above, in most cases newer versions work fine, but I can't guarantee the results. FFMPEG svn builds especially, I've found svn8891 or svn8797 stable on all my systems. If you want to try the latest ffmpeg builds, you'll need to change the codecs in the scripts (the names have changed) acc -> libfaac and h264 -> libx264.

Installation

  1. Install DVD Decrypter.
  2. Create a folder called C:\Program Files\MP4 and extract the contents of FFMPEG, AAC Gain, MP4 Box, BeSweet, Atomic Parsley and the MP4 Video Scripts into it.
  3. Add C:\Program Files\MP4 to your path. Right click on My Computer, Properties, Advanced, Environment Variables, System Variables, Path, Edit, Add ";C:\Program Files\MP4" (without quotes) to the end of the list.
  4. Create a folder under My Documents\My Videos\_Import to save your working video files into. This can be a different location if you wish, of course.

Overview

These are the steps we will do for each video:

  1. RIP: The VOB file from the DVD
  2. PREVIEW: To calculate the frame cropping.
  3. ENCODE: Convert the VOB into an MP4 H.264 File, Normalize and Remux.
    Video is encoded using anamorphic wide screen*, normalizing makes every movie playback at a consistent volume (normalizes to 89db, consistent with MP3Gain for music files), and the remuxing is necessary to convince iTunes your iPod can actually play the file.
  4. TAG: Add the meta data (tag) to the file such as the Title, Year, Genre etc.
  5. COVER ART: Optional —Add DVD cover for movies or the TV show title frame so it looks nice in iTunes.

* This is so the picture completely fulls the screen of the iPod, but when played on a wide screen TV (set to FULL mode) the movie will be restored to its original wide screen format.

TIP: iPod's currently don't support the files with a SAR (Sample Aspect Ratio) of anything other than 1:1. In future hopefully this will be fixed. I suggest including the movie's Aspect Ratio in the description tag, so if in future other SAR's are supported, you can use this information to calculate the appropriate SAR for the movie and update the file's MPEG headers.

UPDATE: If you wish to use these files with an AppleTV, which does support custom SAR's, you can use Quicktime format to save a linked movie with the correct SAR (and subtitles as well). In Quicktime Pro, select Show Movie Properties, Video Track and Visual Settings. Uncheck Preserve Aspect Ratio and enter appropriate resolution. For 2.40:1 (720x320), 2.35:1 (720x328), 2.20:1 (720x350), 2.00:1 (720x384), 1.85:1 (720x416), 1.78:1 (720x432), 1.37:1 (720x562). These are for PAL, for NTSC divide the height by 1.2. The picture should resize, and then Save As.. a .mov reference movie. This file will play in correct widescreen format on an AppleTV and should only be about 1MB in size (it uses the original MP4 file for the picture and sound content).

1. RIP: The movie using DVD Decrypter

DVD Decrypter should be set to IFO mode and have the following settings chosen—for screens not shown below the defaults are fine. These settings will auto select the primary language of the DVD (i.e. English, unless it's a foreign language film). If it's a foreign film and you want the English soundtrack, use the stream processing tab in DVD Decrypter to check that language and uncheck the foreign one.

Set the destination folder on the DVD Decrypter main screen to be the My Documents\My Videos\_Import folder created earlier (click on the yellow folder icon)

Once you have all these set, put in your DVD movie disc, and click the "DVD > Hard Disk" at the bottom left. DVD Decrypter will then rip the DVD to your hard disk as a VOB file. Open the windows explorer, navigate to the _Import folder and you should see a file named something like VTS_02_PGC_01_1.VOB (the numbers may be different, but that's ok). Rename this to something more useful, e.g. My Movie.vob where "My Movie" is the actual title of the film.

TIP: An example extract script is included that shows how to rip multiple episodes from a TV Show DVD in one step. You will need to edit the VTS and PGC numbers to match the DVD in question, and you could use multiple blocks with a ECHO Insert Disc X and a PAUSE before each to process a whole season.

2. PREVIEW: Calculate the frame cropping.

In the _Import folder create two batch files called preview.cmd and convert.cmd. We'll use convert.cmd to convert the MPEG2 VOB files into an MPEG4 files (the format that the iPod needs), and the preview one to do a similar conversion but to generate a preview instead.

Edit your preview script and call the mp4preview.cmd script for each VOB file, use an output file similar to: Movie - Preview.mp4. This generates a rough 10sec conversion at 100% without cropping in H263 format (fast). Play each resulting file, and find a frame where you can clearly see the edge of the picture. Press Print Screen and paste into Photoshop (or another program with measuring tools) and measure the distance between the inner edge of Quicktime player window and edge of the video content (overlap the into the content area by 1px). Measure the for all 4 sides.

Using the crop calculator webpage included to generate the scale parameters variable (select the Source frame size and enter the four measurements taken earlier, click calculate and copy & paste the output into your convert script). There is an example convert.cmd in the examples folder, have a look at this for the syntax. At this point you can delete the preview MP4 files.

You'll need two lines in convert.cmd for each video file you wish to convert. The first one sets the cropping (scale params) and the second one calls the encode script. Converting this way lets you process many files at once, good if you want to set it going overnight.

TIP: When working with MPEG4 files, always use the standard "MP4" file extension, and not the non-standard Apple "M4V" as this can cause problems. M4V is meant to be for raw video stream and not a video and/or audio stream inside a MPEG4 container.

TIP: The MP4 file's name must be less than 64 characters (including the extension) or Quicktime for Windows will say it's corrupt.

3. ENCODE: Convert the movie to H.264 format

After the scaling parameters SET command, your convert script will call one of the 3 encode scripts. These scripts use slightly different video / audio settings:

  • MOVIE: Encodes with the highest video quality and audio quality. Preprocesses the soundtrack with BeSweet to downmix 5.1 Dolby Digital into 2.0 Dolby Surround 2 this multiplexes LCR+Surround into the Stereo output (use the "Dolby Pro Logic II Movie" on your AV receiver during playback).
  • TVSHOW: Encodes with slightly lower video and audio bitrates. TV Show material is often easier to encode and volume of data is higher, so these settings are a good trade off of quality vs. disk space.
  • MUSIC VIDEO: Uses the same video settings as the TVSHOW script, but with very high audio quality

If you are unsure of the settings any of the encoding scripts, just run them without without parameters to see the help screen.

Run the convert script by starting the Command Prompt, changing into the _Import folder, and type convert and press ENTER. This will take a while... ;-)

TIP: Interlaced TV Shows and mono soundtracks require additional parameters, see the help screens of the encode scripts for further information.

4. TAG: The MP4 files with Atomic Parsley

In the _Import folder create script called tag.cmd. There is an example tag script included, so use that as an example. The tag scripts also print help screens, so you'll want to refer to those too. This tagging can be done manually in iTunes but it's really slow as the whole file must be rewritten. Atomic Parsley has to do this also, however here you can batch several files at once saving a lot of time.

Run the tag script by starting the Command Prompt, changing into the _Import folder and typing tag and then pressing ENTER.

TIP: Descriptions/synopsis's longer than 255 characters won't display in iTunes. Watch out for international characters (accented western, asian and eastern) the command prompt uses a different code page than iTunes, so you'll need to correct these using iTunes afterwards.

5. COVER ART: Add the Cover Art (optional)

This is really an additional tagging step, however I find it easier to separate this out from the main tagging. This adds the artwork displayed in Album and Cover Flow view in iTunes and on the iPod's screen in TV Out mode. For movies I use the front of the DVD Cover (you can scan in the cover yourself, or download it from one of the sites on the Internet (just google "DVD Covers"). For TV Shows I normally use the title frame from the show, same procedure as in the preview step above except you crop and save a jpeg. Use the mp4tag-artwork script in a batch file similar to your tag one.

6. ADDITIONAL STUFF: Subtitles etc

There are scripts included for processing subtitles. However, bear in mind the iPod currently doesn't support subtitles, so like adding the Aspect Ratio/SAR to the tags, adding these will be possibly be useful in the future but not so much now. If you add subtitles, you must do it before the tagging step (any existing tags aren't copied to the new [output] file). To rip the subtitles to a text file, you can use SubRip in SubPictures to Text via OCR mode, you'll also need to make sure you check the appropriate subtitle stream in DVD Decrypter during the rip stage.

TIP: If you rename the file MP4 to have the extension .3gp, Quicktime for Windows will display the MP4 timed-text subtitles.

UPDATE: Subtitles are supported in Quicktime containers, .mov files, in Quicktime Player, iTunes 7.4, AppleTV and iPod 6G (sorry not 5G with 1.2.1). To correctly add the subtitles, see the paragraph about Quicktime containers in the Overview section above, and just before saving the file do this. Open up mp4-subtitle-convert.html, which is included in the latest scripts archive, and paste the contents of the .srt into the top box. Click convert and copy & paste the results from the bottom box into a .txt file. In quicktime open the .txt (you'll need to change the file type to all files) and do a Select All and Copy. Change to the main movie window, make sure you're at the very start of the film, and choose Edit > Add to Movie (not Paste). In the Movie Properties dialog, select the Text Track, Visual Settings and set the Offset to 0 x 430. Close the movie properties box, and do the Save As... Reference Movie as normal. Your linked Quicktime file should now have subtitles and the correct aspect ratio! ;-)

Once you're finishing tagging the files, move them to their final storage location and then delete the VOB files (you can also delete the video-stream.mp4 & audio-stream.mp4 working files and FFMPEG logs). Import the finished movies into iTunes as normal, sync to your iPod and enjoy!

Great hall locations: Music , iPod  |  Talked about by: 2 Fraggles

February 15, 2006

New iPod Lovin'

I've recently upgraded to a 60GB Video iPod from my ancient 2nd generation ipod, and am absolutely loving the thing. I've finally got 'on-the-go' playlists, shuffle songs that is actually random, and video functionality (which i hadn't really considered) turned out to be pretty damn cool.

After purchasing the apple a/v cable, I'm in the progress of converting a huge amount of video that i'll be able to watch anywhere. Well actually, i've converted some of it several dozen times so far. Finding the best encoder, codec, and settings turned out to be a little challenging, since i wanted pretty good quality on a TV *and* a reasonable file size—2500kbps was out of the question.

In the end i decided on ffmpeg (windows binaries) as the encoder with these settings:

Video

-vcodec xvid -s 512x384 -qscale 6 -g 300
The "xvid" codec turned out to produce much cleaner and smaller files than the "mpeg4", and plays on the ipod fine. Depending on content, I've increased the quality a bit, levels 4 and 5 are a fair bit smoother though file size jumps up considerably—I've done some of my faster moving music videos at these levels.

Audio

-acodec aac -ac 2 -ar 32000 -ab 48 -async 1
High passing frequencies above 16kHz and lowering the bitrate saves a huge amount of space. TV shows especially could go even lower, however I'm using shure e3c's and am quite sensitive to audio quality, and these were the lowest settings i was happy with. Incidentally, with the apple stock headphones 22050 & 32 or 40 was passable since they seem to roll off the top end anyway.

Misc

-f mp4
Makes sure the muxing is suitable for the ipod.

Once you've got the video converted, tagging it so the iPod sorts it nicely is another matter. iTunes [as of this writing] is completely unable to tag video files as TV Shows properly. It has a TV Shows setting, but it is ignored by the iPod. Fortunately there's a tool called Atomic Parsley, which was originally just for the mac, but windows as well.

One thing I've found really handy to do with my video tagging, is to put "Videos" in to the "Album" field for all my videos; movies, music and tv shows. The reason is iTunes treats videos just like songs and finding a handful of videos intermixed with them is a pain. With the album set to videos, you can just type "videos" in the search field to list all the videos on your iPod—sweet.

Here is an example of Atomic Parsley config I use, for a TV show:

--genre "TV Shows" --stik "TV Show" --artist "Seinfeld" --album "Videos" --year 1989 --TVShowName "Seinfeld" --TVEpisode "The Seinfeld Chronicles" --TVEpisodeNum 1 --TVSeasonNum 1 --title "The Seinfeld Chronicles" --writeBack

To get the video off your DVD's the best tool is DVD Decrypter, you'll have to google search for it as Macrovision (the makers of DVD encryption) under threat of lawsuit made them take the site down and remove the software. The last version was 3.5.4, and make sure you uncheck the "Check for Program Updates" under Settings > Events.

From the UI you can't extract more than one episode at a time in IFO mode (the mode you want to be using). However, from the console it's possible to write a batch script to extract them one after the other. Just add one line for each PGC with the switches below:

/MODE IFO /DEST F:\VIDEOS /VTS 1 /PGC 1 /START /CLOSE /SPLIT NONE /DIRECT 0x80 0xE0 /NAMING PGC

In closing, having all my video with my me is just as cool as having all my music. iPod's just seem to get better and better!

UPDATE: The information above has been superseded, please see the more recent post entitled: Step By Step Guide: Putting DVD's On To Your iPod

Great hall locations: Music , iPod  |  Talked about by: 1 Fraggles

May 9, 2005

Shitty MP3's Must Die!

Dr Evil

With my next overseas trip just over a week away, I'm in the final mad rush to get everything ready, including breaking in my new Shure e3c Noise Isolating Earphones. These are the in ear canal style units that musicians use, and I have to say the sound quality and detail is quite extraordinary!

I used Sony active noise canceling headphones on my last trip. They helped a lot in reducing the aircraft engine noise, but were quite bulky, only worked on low/mid frequencies and need batteries. The Shure ones on the other-hand block about 25dB across all frequencies (the noise-canceling ones were 10dB) and are very compact, which will help my carry-on baggage only plan for this trip.

They take about a week to properly mold to your ear canal (or your ear canal to mold to the earphones as the case maybe, lol) and break-in the speaker drivers. We're at day 3 now and they're definitely getting more comfortable to wear—and the sound quality still blows me away!

Shure e3c Earphones

However, this does cause one major problem— 128 & 160kbps CBR MP3's which previously sounded ok, now sound terrible. The encoding artifacts are clearly audible in all their hideous glory. The 160kbps songs sound flat and lifeless, and we won't even mention mangled state of the poor 128k ones, lol.

So, I've been on a mission hunting out all the poor quality songs in my collection with the help of Tag & Rename, which is an excellent program.

Tag & Rename can also tell you which encoder was used to make the file (this is separate from the ID3v2 tag's "encoded by" field), which is quite handy for generating a report of all the MP3's not encoded with LAME and are CBR 160kbps or less.

On a related note, I checked with the airline the other day and my skateboard is good to go as carry-on. It was the only piece of baggage I thought might possibility have to be checked, which would ruin my carry-on only plan. Apparently if my main carry-on bag isn't too big, it's all good ;)

Update: It appears I was missed informed, and the skateboard can only be taken on as checked luggage—which added just under an hour to me getting out of LAX the other day, arrrghh!

Great hall locations: Music , iPod  |  Talked about by: 0 Fraggles