Keeping up with iplayer-dl

I’ve had a productive evening hacking iplayer-dl. Here’s a short list of improvements:

  • Checks the availability of different versions of the video based on type and date.
  • Fetches the ‘Original’ version by preference. (Previously, it would tend to fetch the sign language version. This can be modified in the code if you do want signed videos.)
  • Resumes downloading if interrupted and restarted.
  • You can now specify just a PID (the eight letter/number code) on the command line instead of the URL, if you prefer.
  • Uses the XML metadata service to determine the title and subtitle with greater accuracy.

And, as I promised, here’s a Subversion repository for those of you who want to keep up to date with changes:

svn co http://paulbattley.googlecode.com/svn/iplayer-dl

The latest version will continue to be available via the download link.

There’s an update here.

2008-04-17 01:19 UTC.

Comments

  1. Oliver

    Wrote at 2008-04-17 10:49 UTC using Firefox 2.0.0.13 on Linux:

    I just want to thank you for creating this – it’s amazingly useful, especially for those of us who:
    * Use Linux, where Adobe Flash is absolutely terrible unless you like small windows and frequent crashes
    * Firewalled, so the streaming doesn’t work anyway

    Your script works like a dream, and has inspired me to give Ruby another go.

  2. Terence Eden

    Wrote at 2008-04-17 13:49 UTC using Firefox 2.0.0.14 on Windows XP:

    Just to let you know, this runs really well on the Nokia Internet Tablets (N810). It’s a pity that the videos need to be run through ffmpeg/mencoder before they’ll play on the NIT stutter free!

    Keep up the good work :-)

  3. kzap

    Wrote at 2008-04-18 16:17 UTC using Firefox 2.0.0.6 on Linux:

    I’m sorry to say it’s not working for me it comes up with
    The BBC iPlayer appears to have changed.
    Please look for an updated version of this script.

  4. Paul Battley

    Wrote at 2008-04-18 16:20 UTC using Firefox 3.0b5 on Linux:

    kzap: Which programme? It’s still working on Newsnight.

  5. Math Campbell-Sturgess

    Wrote at 2008-04-18 18:00 UTC using Safari 525.13 on Mac OS X:

    Hi there, I’ve tried posting on here a few times, and it’s still not showing up :(
    trying now using Safari rather than Firefox.

    Just wanted to thank you for your script, and tell you I’ve made a Mac OS X gui-based app using it so Johnny User who doesn’t like command-lines can still use it.
    Now of course, having noticed you’ve updated to do new things like resume downloads and better capturing, I’ll have to go back to work on it, but at least the first version works, roughly.
    I’ll upload it as soon as I can find somewhere, since I’ve tried MacUpdate.com but they won’t seem to do it (I’m guessing in case the BBC take umbrage at them). Soon as it is I’ll put the source up too, so everyone can have a good laugh at my woefully inept programming (this is only my second real program!).

    Thanks again, and hope you don’t mind me using (and editing; had to change the percentage-out bit to put onto the stderr instead of stdout, only way I could get the data for some reason) your script. I have credited you for the script :)

  6. Paul Battley

    Wrote at 2008-04-18 19:58 UTC using Firefox 3.0b5 on Linux:

    Math, sorry about the posting troubles. My spam-catcher script thought that your comments looked spammy, so it was hiding them.

  7. Paul Battley

    Wrote at 2008-04-18 20:02 UTC using Firefox 3.0b5 on Linux:

    Putting the status output on stderr is probably something I should be doing anyway. If there’s anything like a command-line option that I could offer to make GUI interoperability easier, I’d happily add it.

    I’m willing to host the package for you if you can’t find anywhere.

  8. Math Campbell-Sturgess

    Wrote at 2008-04-18 22:21 UTC using Safari 525.13 on Mac OS X:

    No bother. Spam filters can be tricksy…

    I don’t think there’s anything else you can do that’ll make it easier for me now, although putting stuff on the error console rather than the standard output might help.
    I think why I was having so much trouble was that your script was doing something to erase the last line outputted (e.g “5.1% title”) with the next line, and I think my code wasn’t getting the data before it was replaced by the last one. I’m not sure since I’ve never touched ruby before, and I’m fairly new to programming.
    Your script refers to “if $stdout.tty?” then $stdout.flush. This function was the bit that stopped it all working in the end.
    I just replaced that if statement bit with a single line that put the percentage followed by the title to the stderr in the format stderr.puts ”#{ percentage }, #{ title }”. The comma is more important than it looks, because on the Cocoa side of the task, the data comes in from the error console, gets assigned to a string, then the string gets divided up into substrings based on the seperator ”, ”. This allows me to separate the percentage from the title, then use the double from the % status to animate a progress bar.

    If you could host it, that would be great. I looked into getting a googlecode, but if you’ve already got one, we could use it to gather together the various methods of accessing the iPlayer…

    Math

  9. Tim Rowe

    Wrote at 2008-04-19 07:12 UTC using Firefox 2.0.0.13 on Linux:

    All the downloads I’ve tried recently come back with the error: “Not available as MP4”

  10. kzap

    Wrote at 2008-04-19 08:45 UTC using Firefox 2.0.0.6 on Linux:

    I tried downloading the last Torchwood and the last to Dc Who’s.
    I’ll try newsnight just to check if I am doing any thing wrong.

  11. kzap

    Wrote at 2008-04-19 08:56 UTC using Firefox 2.0.0.6 on Linux:

    For news night I got
    “This program is currently not available in MP4 format” or something like that.

    I did manage to start to download an episode of the Apprentice which is a shame because I don’t acually watch the Apprentice but at least it is working on something.
    Well done on this script I’m using linux (Ubuntu) and am sick of having to watch everything online.

  12. Paul Battley

    Wrote at 2008-04-19 10:05 UTC using Firefox 3.0b5 on Linux:

    I think I’ve fixed the bug that was stopping the downloads working for some programmes. It’s all to do with the flakiness of the way I’m parsing the JavaScript that defines the versions. I need to make a more reliable parser.

  13. kzap

    Wrote at 2008-04-19 10:34 UTC using Firefox 2.0.0.6 on Linux:

    Seems to be working now I don’t know what you did but it works.
    I’ve managed to get a Top Gear and am downloading the latest Dc Who thanks a lot.

  14. Tim Rowe

    Wrote at 2008-04-19 10:54 UTC using Firefox 2.0.0.13 on Linux:

    With the most recent, version I’m still getting the “Not available in Mp4” error – unlike the above poster :(

  15. Paul Battley

    Wrote at 2008-04-19 10:57 UTC using Firefox 3.0b5 on Linux:

    For error reports, please give the URL or PID of the programme you can’t download.

  16. kzap

    Wrote at 2008-04-19 11:08 UTC using Firefox 2.0.0.6 on Linux:

    Got the not available in MP4 error on this
    http://www.bbc.co.uk/iplayer/page/item/b0074gkn.shtml
    Not on the others I downloaded though.

  17. Tim Rowe

    Wrote at 2008-04-19 11:22 UTC using Firefox 2.0.0.13 on Linux:

    “No MP4” error on trying this URL:

    http://www.bbc.co.uk/iplayer/page/item/b009zh0p.shtml

  18. Simon

    Wrote at 2008-04-19 12:01 UTC using Safari 525.13 on Mac OS X:

    Paul, thanks so much for this script.

    Tim: that Later with Jools Holland doesn’t work on the iPhone either – so maybe there really isn’t an MP4 version?!

  19. Simon

    Wrote at 2008-04-19 13:19 UTC using Safari 525.13 on Mac OS X:

    If only the quality of the MP4 files were just slightly better, then this solution would be perfect! Paul, do you know if the Wii files are a higher resolution?

  20. Tim Rowe

    Wrote at 2008-04-19 13:21 UTC using Firefox 2.0.0.13 on Linux:

    Simon – Thanks for that. I’ll try some other d/loads (mp4) to see if they work.

    Cheers.

  21. kzap

    Wrote at 2008-04-19 13:29 UTC using Firefox 2.0.0.6 on Linux:

    I’d think the wii files would be better as they are meant for a bigger screen personally I don’t think the MP4’s are that bad they look about the same as the online streaming quality.
    I mean I plan to burn some to a disc so I can watch them on my big screen without having to worried about all this S-Video cable nonsense.

  22. Child

    Wrote at 2008-04-19 13:43 UTC using Firefox 3.0b5 on Linux:

    http://www.bbc.co.uk/iplayer/page/item/b009vsbp.shtml?src=ip_lc

    Currently giving: This programme is not currently available in an MP4 version.

    Thanks for a great script though please keep up the good work.

  23. Simon

    Wrote at 2008-04-19 15:43 UTC using Safari 525.13 on Mac OS X:

    OK, ignore my earlier Wii comment/question. It’s not anything like a downloadable MP4 file, they’re using Flash 7/Sorenson Spark streaming video to deliver to Wiis. Let’s hope BBC open up to Apple TV, then we might see better MP4 files available. I might even buy an Apple TV. :-)

  24. Paul Battley

    Wrote at 2008-04-19 16:10 UTC using Firefox 3.0b5 on Linux:

    Top Gear and Later with Jools Holland genuinely aren’t available as MP4 files.

    In Search of Medieval Britain should work: that’s a bug, and I know why it’s happening.

  25. Paul Battley

    Wrote at 2008-04-19 16:41 UTC using Firefox 3.0b5 on Linux:

    I’ve fixed the bug that stopped In Search of Medieval Britain working.

  26. Math Campbell

    Wrote at 2008-04-19 17:19 UTC using Firefox 3.0b5 on Mac OS X:

    Don’t know if this will make it past your spam-filter (I think it’s my name, damn my parents!) but I’ve created a source-forge account for my gui-frontend of your script, and also MacUpdate finally decided to honour me by hosting the download, finally.
    Sourceforge is still pending, but the link is http://sourceforge.net/projects/iplayerdownload/
    and MacUpdate is live already, on http://macupdate.com/info.php/id/27354/iplayer-downloader

    As soon as Sourceforge is up you can download the source so’s you can see what I did to allow your script to work with my (very badly coded) gui.
    I’m gonna integrate a newer version fo your script as soon as I can but it may be a few days since I have a uni hand-in in 2 weeks that I’ve not really started on yet…

  27. Simon

    Wrote at 2008-04-20 13:25 UTC using Safari 525.13 on Mac OS X:

    Math, your application looks really interesting. I plan to download it later and have a look at it.

    I have created some handy AppleScripts (for my own use) to download content, and just thought others might like to use them as well, so I have made them available on Google Code. If anyone would like some handy AppleScripts to trigger iPlayer downloads directly from the Apple menu bar, please check out this page:
    http://code.google.com/p/iplayer-applescripts/

    It certainly isn’t as pretty as your solution, Math (I am no application coder!), but it’s just a different way to trigger the downloads which others may like to use.

    Please note: this is for MacOS X only. Please make sure you read the README.txt file first!

  28. Fireice

    Wrote at 2008-04-20 20:56 UTC using Firefox 2.0.0.14 on Windows XP:

    It seems that they changed something. Revision 111 produces:
    “The BBC iPlayer appears to have changed.
    Please look for an updated version of this script.”
    on every video that I tried.

  29. Paul Battley

    Wrote at 2008-04-21 11:32 UTC using Firefox 3.0b5 on Linux:

    Fireice, it’s still working for me. Please tell me a specific programme that you can’t download.

  30. Jonathan Duncan

    Wrote at 2008-04-21 23:06 UTC using Firefox 2.0.0.14 on Windows Server 2003:

    I’ve been experimenting with this, the iPlayer for iPhone seems to be going some changes, a whole load of programs do not work for the iPhone atm. Not just iPhone scripts, but my actual iPhone (well iTouch) in my hand.

    The erroring shows will return a 404 rather than the 403 which usually signifies a faulty request from the script.

    Examples of shows not working on my iPod touch (nor script):
    http://www.bbc.co.uk/iplayer/page/item/b00b09vp.shtml
    http://www.bbc.co.uk/iplayer/page/item/b00805q2.shtml
    http://www.bbc.co.uk/iplayer/page/item/b0087fmw.shtml

    As far as i can work out its only videos very recently released (pretty much everything on “Evening” for today isnt working but older things are, so perhaps there is some sort of transcoding delay? or perhaps this is the start of them wiping stopping iphone iplayer due to DRM worries (i hope not)

  31. Elliot Smith

    Wrote at 2008-04-21 23:34 UTC using Firefox 2.0.0.2 on Linux:

    Paul, I’ve integrated this into some Ruby code I wrote to provide a Linux UI for iPlayer. Posted about it here: http://townx.org/blog/elliot/ruby-tuesday-miplayer-some-iplayer-ruby-shizz-linux. Thanks for releasing it. What’s the licence?

  32. Paul Battley

    Wrote at 2008-04-22 08:00 UTC using Firefox 3.0b5 on Linux:

    Jonathan: I think there must be a transcoding delay, because I can now download the programmes you listed.

    Elliot: The whole of my Google Code repository is under MIT, which is liberal enough that you should be able to use it with anything. If there’s anything I could do with the output that would make it easier to integrate, let me know.

  33. Elliot Smith

    Wrote at 2008-04-22 08:12 UTC using Firefox 2.0.0.2 on Linux:

    Hello again Paul. MIT licence is great – I’ll put my code under the same licence. Your script is perfectly useful as is. Have you thought about wrapping it into a class and breaking the methods up a bit? That would make it easier for others to integrate. I’ve kind of done this myself for my code, but haven’t taken it as far as I could.

    Thanks again.

  34. rak

    Wrote at 2008-04-22 08:35 UTC using Firefox 2.0.0.14 on Mac OS X:

    Another Mac OSX GUI, iplayer-dl wrapped up in an Applescript package – grab it here: http://labsix.net/beeb/

  35. Tim Rowe

    Wrote at 2008-04-23 09:15 UTC using Firefox 2.0.0.14 on Linux:

    Now able to download the latest “Later with Jools Holland” as a mov. :)

  36. Daniel Hardy

    Wrote at 2008-04-23 12:01 UTC using Firefox 3.0b5 on Mac OS X:

    Paul, anyway to change where the files get downloaded to on your computer?, a small thing I know but it would be useful.

  37. Simon

    Wrote at 2008-04-23 17:34 UTC using Safari 525.18 on Mac OS X:

    Daniel, they get downloaded to the current folder. So if you run the script from ~/Movies, they’ll get downloaded to your movies folder.

    You might find this interesting:
    http://code.google.com/p/iplayer-applescripts/
    It sets up a folder under Movies called “iPlayer” and downloads into that. You can customise the AppleScripts to download elsewhere if you prefer.

  38. rak

    Wrote at 2008-04-24 05:19 UTC using Safari 525.18 on Mac OS X:

    Daniel, With a little more hackery, I’ve updated iPlayer Downloadr so that it prompts you for the download folder, grab v1.2 here:
    http://labsix.net/beeb/

  39. mat

    Wrote at 2008-04-24 11:25 UTC using Firefox 2.0.0.13 on Mac OS X:

    Just tried to download Blood, Sweat and T-Shirts (http://www.bbc.co.uk/iplayer/page/item/b00b0xsh.shtml) from Germany with Vidalia/Tor and an british exit, using iplayerdownloadrv (for OSX), which relies on the iplayer-dl – but it doesn’t work. It doesn’t neither for other episodes, as for example The Apprentice (http://www.bbc.co.uk/iplayer/page/item/b00b0cs1.shtml).

    Is it because the script doesn’t work any longer or might it be a problem with the proxy? Can I redirect the script on the terminal to get it through Tor?

    Thanks!

  40. Math Campbell

    Wrote at 2008-04-24 15:56 UTC using Firefox 3.0b5 on Mac OS X:

    hey mat:
    I just tried both those URL’s in iPlayer Downloader (my gui-based app, not the apple-script one rak does; Gods we need to come up with better names here!!) and they both seem to work.
    Dunno if the BBC are just being slow at updating, cause I tried the blood sweat and tears one last night and had no joy, but it works now, so I’d guess you just have to wait a while before the right file is available.

    The next version I release of my app may have proxy-setting ability; someone’s asked me to put that in over on the sourceforge (http://sourceforge.net/projects/iplayerdownload/) page, and I think I probably can thanks to Paul’s coding work on the script (again, many thanks to The Paul! He da man!). This might resolve your proxy problems, although nothing’s guaranteed.

    Math Campbell

  41. rak

    Wrote at 2008-04-24 20:06 UTC using Firefox 2.0.0.14 on Mac OS X:

    Math, you are right about the names!
    I’ve changed the name of my GUI to: Beeb Downloader.app
    grab v1.3 here: http://labsix.net/beeb/

  42. mat

    Wrote at 2008-04-25 08:36 UTC using Firefox 2.0.0.13 on Mac OS X:

    Well, I could do the proxy-trick with adding a tiny line, so for all the foreign bbc-lovers, here’s how to get the ruby script working:

    Add

    ENV[‘http_proxy’] = ‘http://213.177.255.60:80’

    before line

    if http_proxy = ENV[‘http_proxy’]

    to my iplayer-dl script and though it’s got the speed of a snail – it works!

    Am still wondering if it’s faster with vidalia/tor as proxy (127.0.0.1:8118), but got no result yet…

    And: I can’t preview the mov with VLC while downloading. Is that allright?

  43. rak

    Wrote at 2008-04-25 18:26 UTC using Firefox 3.0b5 on Mac OS X:

    If you want to know if and when the MP4 version is available, check out this hint

  44. Math Campbell

    Wrote at 2008-04-26 01:48 UTC using Firefox 3.0b5 on Mac OS X:

    Cool, I think I can implement that as a preference setting in my app, maybe even putting in a list of proxies for people to choose from (although how I’d keep a list like that current I don’t know, not to mention some sort of testing method so you know they’re online. I guess getting the proxy to resolve google.com or something similar would do, right?).

    Mat; I didn’t mean for you to go chaning names or anything! It’s cool cause it means I don’t have to bother (I am rather lazy) but maybe we need a site to get together on for us all to work stuff out on.
    All kudos to Paul for letting us blether on about this on HIS site, but maybe we should go to a dedicated forum rather than fill up his blog with it! Anyone any ideas?
    I know the beebhack wiki is one place, but it’s not really got a forum; wiki’s are good for articles but lame for discussions…

  45. David Tennant

    Wrote at 2008-04-26 20:24 UTC using Mozilla 1.8.1.8 on Linux:

    Can’t d/l Dr Who
    http://www.bbc.co.uk/iplayer/page/item/b00b3z7h.shtml
    as the script is returning ”...not available in mp4”
    but at http://www.dracos.co.uk/play/bbc-iplayer-mp4/ is saying
    “Download: 26/4/2008 17:17:07pm – 3/5/2008 18:04pm”

    Any ideas?

    Thanks, David

    PS – Anyone seen Rose?

  46. Paul Battley

    Wrote at 2008-04-26 20:33 UTC using Firefox 3.0b5 on Linux:

    David, I think the Dracos script is wrong: there are no MP4 download details in the source of the page at the moment.

    It typically takes a day or so for the MP4 versions to make their way out.

  47. rak

    Wrote at 2008-04-27 01:11 UTC using Firefox 3.0b5 on Mac OS X:

    Math sure comes up with some great ideas(!)...
    Math, Paul & others: I’ve setup a ‘Vanilla’ forum on my site, you are welcome to use it to discuss iplayer-dl-based apps: http://labsix.net/beeb/talk/ – hopefully it will take some load off Paul’s comments section.

    Right now there are 3 categories, ‘General’, Cocoa apps and Applescript apps. If anyone wants categories for other platforms, leave a reply in the ‘General’ discussion. I hope it’s useful to you guys. Thanks again Paul.
    -rak.

  48. kzap

    Wrote at 2008-04-27 07:40 UTC using Firefox 2.0.0.6 on Linux:

    I can’t download the latest Doctor Who either last time I found that if you just waited a day it worked but it’s still not working.