Refactoring iplayer-dl

Using Windows? There’s now an easy graphical downloader.

I’ve done a bit of work on refactoring iplayer-dl, with two aims in mind:

  1. Improving the JavaScript parsing that determines available versions
  2. Making the code easier to repurpose and integrate into GUIs etc.

The most important result of this work is that it’s no longer a standalone script. This shouldn’t necessarily make it harder to use, though. Once you’ve unpacked the code, you’ll need to run the following commands in the iplayer-dl directory to install the libraries and command-line tool:

ruby setup.rb config
sudo ruby setup.rb install

You can omit sudo on Windows, I think; some Linuxes might require you to use su instead.

The iplayer-dl command should be installed somewhere in your path, and you’ll be able to use it from anywhere just by typing its name.

The tarball conforms to the Debian packaging guidelines for Ruby applications, incidentally, so it should be straightforward to make a .deb for Ubuntu/Debian, or even a port for MacPorts.

You can download a snapshot, or check out the latest version:

svn co http://paulbattley.googlecode.com/svn/iplayer-dl

If you’re reporting a bug, please also write the URL or PID of the programme you were trying to download.

2008-04-27 18:59 UTC. Comments: 95.

Comments

  1. Math Campbell

    Wrote at 2008-04-27 19:26 UTC using Firefox 3.0b5 on Mac OS X:

    Cool, dude!

    And I was almost finished with version 1.2! Ack well, the users can wait till I can put this version in! :)

    Couple of questions. I’ve just (briefly) skimmed over the new script(s). Interfacing-wise, the new script IS better (cheers), but to get it going, users will need to use your install script; fair enough, but where does the iPlayer script itself go, and where does everything else go? Does it all go inside it’s own bundle next to the script, or are thins pushed into the main filesystem. If so, do you know if any of your install directorys will need admin rights to be touched?
    I’m thinking I can just have a run-once app to duplicate your install.rb, but if not, I can run yor install.rb programmatically; if so though, I’ll need to get admin rights off the user to run it first…

    Thanks for the updates! Things just keep getting better and better!

  2. Tim Rowe

    Wrote at 2008-04-27 20:02 UTC using Firefox 3.0b5 on Linux:

    Hi,

    Probably something I did incorrectly, being a novice (although I followed the instructions and AFAIK got no error messages)

    But I get this when trying to d/load:

    /usr/local/lib/site_ruby/1.8/iplayer/browser.rb:3: uninitialized constant Net (NameError)
    from /usr/local/lib/site_ruby/1.8/iplayer.rb:2:in `require’
    from /usr/local/lib/site_ruby/1.8/iplayer.rb:2
    from /usr/bin/iplayer-dl:9:in `require’
    from /usr/bin/iplayer-dl:9

  3. Roberto

    Wrote at 2008-04-27 20:06 UTC using Firefox 3.0b5 on Linux:

    I haven’t been able to get the new iplayer-dl script working on my Ubuntu Hardy heron box. Both the snapshot and svn HEAD versions fail for me – invoking like this:

    iplayer-dl http://www.bbc.co.uk/iplayer/page/item/b00b3z7h.shtml

    -gives this error:

    /usr/local/lib/site_ruby/1.8/iplayer/browser.rb:3: uninitialized constant Net (NameError)
    from /usr/local/lib/site_ruby/1.8/iplayer.rb:2:in `require’
    from /usr/local/lib/site_ruby/1.8/iplayer.rb:2
    from /usr/bin/iplayer-dl:9:in `require’
    from /usr/bin/iplayer-dl:9

    -as it happens, I only upgraded to the latest version of the script as I was getting this error using an older version of the script:

    /usr/lib/ruby/1.8/net/http.rb:560:in `initialize’: Connection refused – connect(2) (Errno::ECONNREFUSED)
    from /usr/lib/ruby/1.8/net/http.rb:560:in `open’
    from /usr/lib/ruby/1.8/net/http.rb:560:in `connect’
    from /usr/lib/ruby/1.8/timeout.rb:48:in `timeout’
    from /usr/lib/ruby/1.8/timeout.rb:76:in `timeout’
    from /usr/lib/ruby/1.8/net/http.rb:560:in `connect’
    from /usr/lib/ruby/1.8/net/http.rb:553:in `do_start’
    from /usr/lib/ruby/1.8/net/http.rb:542:in `start’
    from /usr/lib/ruby/1.8/net/http.rb:1035:in `request’
    from /usr/lib/ruby/1.8/net/http.rb:948:in `request_get’
    from ./iplayer-dl.rb:47:in `http_get’
    from ./iplayer-dl.rb:82
    – I’m not sure if this represents another attempted lock-down by the beeb?

  4. Simon Whitaker

    Wrote at 2008-04-27 20:10 UTC using Safari 525.18 on Mac OS X:

    Hi there,

    Nice move! I hit some problems though when I installed the snapshot on my Macbook Pro running OS X 10.5. First problem was that the executable got installed to /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin, which wasn’t in my path. I fixed that with a symlink to /usr/local/bin/iplayer-dl. But now when I run iplayer-dl I get an error:

    /Library/Ruby/Site/1.8/iplayer/browser.rb:3: uninitialized constant Net (NameError)
    from /Library/Ruby/Site/1.8/iplayer.rb:2:in `require’
    from /Library/Ruby/Site/1.8/iplayer.rb:2
    from /usr/local/bin/iplayer-dl:9:in `require’
    from /usr/local/bin/iplayer-dl:9

    Any ideas?

  5. Simon Whitaker

    Wrote at 2008-04-27 20:24 UTC using Safari 525.18 on Mac OS X:

    Ah, looks as though browser.rb is just missing:

    require ‘net/http’

    All the best,

    Simon

  6. Paul Battley

    Wrote at 2008-04-27 20:26 UTC using Firefox 3.0b5 on Linux:

    Sorry about that: I accidentally dropped out the require ‘net/http’ line when refactoring, and, because of my environment, it still worked.

    I’ve checked in a fix, and I’ll update the tarball in a few minutes.

  7. Paul Battley

    Wrote at 2008-04-27 20:33 UTC using Firefox 3.0b5 on Linux:

    OK, done. The snapshot link now goes to iplayer-dl-0.1.1.tar.gz, which should work.

  8. Paul Battley

    Wrote at 2008-04-27 20:44 UTC using Firefox 3.0b5 on Linux:

    Math, in response to your questions:

    It’s not my install script: setup.rb is a quasi-standard script written by one of the core Ruby developers and used by many Ruby libraries to install themselves. Exactly where the files go depends on the system.

    However, if you’re integrating the package, the good news is that you don’t need to install anything: as long as you tell Ruby where to look for the libraries, you can run it in place, e.g.:

    ruby -I /some/path/lib /some/path/bin/iplayer-dl

  9. Winne

    Wrote at 2008-04-27 20:46 UTC using Firefox 3.0b5 on Windows Vista:

    I’m getting this error in Windows:

    c:/Program Files/Ruby/lib/ruby/site_ruby/1.8/iplayer/downloader.rb:32:in `available_versions’: undefined method `inject’ for nil:NilClass (NoMethodError)
    from C:/Program Files/Ruby/bin/iplayer-dl.rb:40

    Any ideas?

  10. Paul Battley

    Wrote at 2008-04-27 20:49 UTC using Firefox 3.0b5 on Linux:

    Winne, which programme?

  11. Winne

    Wrote at 2008-04-27 20:53 UTC using Firefox 3.0b5 on Windows Vista:

    I’ve tried Doctor Who (ID: b00b3z7h) and Young Dracula (ID: b0079cj9).

  12. Paul Battley

    Wrote at 2008-04-27 20:58 UTC using Firefox 3.0b5 on Linux:

    Winne, Doctor Who isn’t available yet. Young Dracula works for me.

    Do you get the same error for both? Which version of Ruby are you using?

  13. Winne

    Wrote at 2008-04-27 21:00 UTC using Firefox 3.0b5 on Windows Vista:

    Yes, I get the same error for both.

    > ruby—version
    ruby 1.8.6 (2007-09-24 patchlevel 111) [i386-mswin32]

  14. Paul Battley

    Wrote at 2008-04-27 21:08 UTC using Firefox 3.0b5 on Linux:

    Winne, what happens if you run ‘rake test’ in the source directory?

  15. Winne

    Wrote at 2008-04-27 21:20 UTC using Firefox 3.0b5 on Windows Vista:

    Paul,

    rake gives me:
    2 tests, 2 assertions, 0 failures, 0 errors

    It looks like it was a flaky network connection on my end. I reset my router and now the program is working great.

    Thanks!

  16. Paul Battley

    Wrote at 2008-04-27 21:22 UTC using Firefox 3.0b5 on Linux:

    Thanks for the update. I was beginning to get worried!

  17. rak

    Wrote at 2008-04-28 17:03 UTC using Firefox 3.0b5 on Mac OS X:

    Paul: Trying to package up the new script into my GUI. I put the lib and bin folder in the Applescript package and referenced them in the script as you mentioned with ‘ruby -I …’ – but I receive a similar error whether I run from the terminal or from the GUI:

    ./lib/iplayer/downloader.rb:57:in `download’: undefined method `[]’ for nil:NilClass (NoMethodError)
    from bin/iplayer-dl:57
    from bin/iplayer-dl:56:in `open’
    from bin/iplayer-dl:56

    Any ideas?

  18. rak

    Wrote at 2008-04-28 17:44 UTC using Firefox 3.0b5 on Mac OS X:

    Sorry, scrap the last comment, it seems to be the programme I was trying, PID: b00b3112 – (Note to self: Test, Test and Test again before opening pie-hole.) A similar error occurs in the previous iplayer-dl too:
    [...] iplayer-dl.rb:154 … undefined method `[]’

    Both errors refer to the line:
    max = response.to_hash[‘content-range’][/d+$/].to_i

  19. Paul Battley

    Wrote at 2008-04-28 19:02 UTC using Firefox 3.0b5 on Linux:

    rak, that’s an odd one. The MP4 file seems to be unavailable. I wonder if it works on iPhones.

    I’m going to put it down to an error on the BBC side for now.

  20. Simon

    Wrote at 2008-04-28 21:47 UTC using Firefox 2.0.0.14 on Linux:

    I’m trying to get the first episode of ‘Out Of The Blue’ but keep getting the message “This programme is not currently available in an MP4 version”. Is this normal? Will it become an MP4 soon?

  21. Paul Battley

    Wrote at 2008-04-28 22:10 UTC using Firefox 3.0b5 on Linux:

    Simon, that’s normal. The MP4 files usually take a while to show up. Give it 24 hours or so.

  22. Simon

    Wrote at 2008-04-28 22:15 UTC using Firefox 2.0.0.14 on Linux:

    Okay. Thanks Paul. I’ll let you know once it is, just in case anyone else is trying to grab the file too.

  23. Simon

    Wrote at 2008-04-30 17:40 UTC using Firefox 2.0.0.14 on Linux:

    Well nearly 48 hours later and still no ‘Out Of The Blue’ episode 1 MP4 download available. Episodes 2 and 3 have got them already. I guess I should be patient and wait, but is there a chance that it will just be overlooked and not have an MP4 download available?

  24. Paul Battley

    Wrote at 2008-04-30 20:52 UTC using Firefox 3.0b5 on Linux:

    Simon, I think it’s genuinely not available. Oversight? Deliberate omission? I don’t know why.

  25. Dmitry

    Wrote at 2008-05-03 10:33 UTC using Internet Explorer 7.0 on Windows Vista:

    Hi,
    Thanks Paul for the job you’ve done.
    I’m having problem with b009wbdl using the last iplayer-dl.

    iplayer-dl b009wbdl
    Resuming download at 0 bytes.
    C:/ruby/lib/ruby/site_ruby/1.8/iplayer/downloader.rb:57:in `download’: undefined
    method `[]’ for nil:NilClass (NoMethodError)
    from C:/Soft/iplayer-dl/bin/iplayer-dl:57
    from C:/Soft/iplayer-dl/bin/iplayer-dl:56:in `open’
    from C:/Soft/iplayer-dl/bin/iplayer-dl:56

  26. Winne

    Wrote at 2008-05-03 23:55 UTC using Firefox 3.0b5 on Windows Vista:

    I am using the program over a UK proxy. It works great but I am probably overloading the proxy server. I think it is possible to get the URL for the MP4 file and start the download through the proxy and then complete download through a direct connection. Any chance you can try this out?

  27. kzap

    Wrote at 2008-05-04 06:35 UTC using Firefox 2.0.0.6 on Linux:

    @ Winne
    I see no reason why not.
    I think it could be a good idea for anyone might want to download the file with another program, like a downloading client.

  28. jimlad

    Wrote at 2008-05-04 08:55 UTC using Firefox 3.0b5 on Linux:

    Great work!

    I was wondering, Is there an easy way to specify a download location. ATM the program ends up in your working directory which means I have to copy it elsewhere.

  29. Math Campbell

    Wrote at 2008-05-04 10:46 UTC using Firefox 3.0b5 on Mac OS X:

    jimlad: there’s a few secondary apps that use Paul’s script to do just that; even I wrote one!
    Have a looksie over at the beebhack wiki for the links. They’re better for people who are less command-line savvy, and allow you to specify directories etc.

    To Paul: still working on getting your latest script integrated; busy trying to get a preferences-pane going to set working directory and proxy settings. Do you think the download could be started using a different client?
    As in, use your script to get the filename, then begin the actual data transfer using an NSURL or whatever?
    If so, do you think it’d need to have it’s user-agent changed too, or can anything grab the file once the real URL has been located..

  30. Paul Battley

    Wrote at 2008-05-04 11:59 UTC using Firefox 3.0b5 on Linux:

    jimlad: I’ve updated the interface script to take various options in response to requests in this thread. One of these is the download location.

    Math: I’m sure that you could use NSURL or something like that to do the final download, but you’d need all the headers (User-Agent, Cookie, Range, etc.). If all you want is feedback on progress, though, you could equally well just watch the stderr output.

    Winne: I don’t think that will work, but I can’t test it.

  31. jimlad

    Wrote at 2008-05-04 20:26 UTC using Firefox 3.0b5 on Linux:

    Paul

    Thanks for that, works great!

  32. RobJN

    Wrote at 2008-05-05 21:55 UTC using Firefox 2.0.0.14 on Windows Vista:

    Hi thanks for the great work. My own skills only went as far as changing browser user agent – the loophole which the BBC quickly fixed.

    Yet until a UI comes along I am being pushed in terms of knowledge (I unfortunately have no programming skills). Can you just explain in layman’s terms to me how this should work. I was ok with downloading ruby and the first version of iplayer-dl, but am confused now. I have ran the commands in the post, but am unable to get it to work unless I cd to C:rubybin and then run “ruby iplayer-dl PID” am i missing something as I thought the install allows iplayer-dl to be used as a command anywhere?

    Thanks

  33. Paul Battley

    Wrote at 2008-05-06 21:23 UTC using Firefox 3.0b5 on Linux:

    Rob, I’m not really sure what the situation is on Windows. Can anyone else help?

  34. mat

    Wrote at 2008-05-07 06:37 UTC using Firefox 2.0.0.14 on Mac OS X:

    Your script has worked with the added ENV variable ( ENV[‘http_proxy’] = ‘http://127.0.0.1:8118’ – tor with british exit nodes) for the past two series of Blood,Sweat & T-Shirts, but today I don’t get it started, not even with the checkouted version with the following error message:

    ruby iplayer-dl b00b6tx7
    /Library/Ruby/Site/1.8/iplayer/downloader.rb:32:in `available_versions’: undefined method `inject’ for nil:NilClass (NoMethodError)
    from iplayer-dl:74

    Any ideas how to get along with that?

    Thanks!

  35. Paul Battley

    Wrote at 2008-05-07 06:48 UTC using Firefox 3.0b5 on Linux:

    Mat, I can download that programme directly and through a proxy, so I suspect one of three issues:

    1. The BBC have started checking the ultimate client location (X-Forwarded-For) as well as the proxy location;
    2. Your Tor endpoint isn’t actually in the UK;
    3. The BBC thinks the endpoint isn’t in the UK based on its IP address.

  36. mat

    Wrote at 2008-05-07 07:32 UTC using Firefox 2.0.0.14 on Mac OS X:

    Hm, tried it with another proxy adress, but didn’t work either. What still is strange that I can watch it on iplayer with tor/vidalia set as mentioned, but the download fails.

    Hey, still got six days to get along, I need that download so badly :-)

  37. mat

    Wrote at 2008-05-07 08:02 UTC using Firefox 2.0.0.14 on Mac OS X:

    Hey,
    actually works when I rip the proxy setting from the script and add it in the terminal command:

    ruby iplayer-dl—http-proxy=127.0.0.1:8118 b00b6tx7

    Works perfect with the checked out version ;-)

  38. Henry Bradley

    Wrote at 2008-05-07 22:41 UTC using Safari 525.18 on Mac OS X:

    Sorry, I’m new to this and cannot get this to work – though it’s bound to be due to my inexperience. I followed the install instructions and it reports back that everything’s fine but when I type iplayer-dl it says there’s no known command. Are there any instructions for use as I have no idea what I’m doing or what I’m doing wrong – thanks. H

  39. Henry Bradley

    Wrote at 2008-05-07 22:57 UTC using Safari 525.18 on Mac OS X:

    I got it to work (I think, it’s still downloading) but only by typing “ruby iplayer-dl PID” from within Downloads/iplayer-dl-0.1.2/bin/ – it won’t work from anywhere else. H

  40. Paul Battley

    Wrote at 2008-05-07 23:03 UTC using Firefox 3.0b5 on Linux:

    Henry, it sounds like wherever Ruby installs to on OS X isn’t in the path—or, at least, not in the path of your setup.

    You’ll have to run it from wherever it is (/usr/local/bin, maybe?) or add that to the path.

  41. Henry Bradley

    Wrote at 2008-05-08 00:56 UTC using Safari 525.18 on Mac OS X:

    It works fine where it is, I’ll worry about the details later. Thanks for a great little tool, works great! H

  42. Henry Bradley

    Wrote at 2008-05-08 07:30 UTC using Safari 525.18 on Mac OS X:

    Is there an iPhone schedule or a way to find out which shows have mp4 streams other than to try the PID’s? At the moment I’m pulling the PID’s from the http links to the show, is that the best way?

  43. Tom Davie

    Wrote at 2008-05-08 18:20 UTC using Safari 525.18 on Mac OS X:

    Hey, thanks for implementing iplayer-dl, but also, please provide a version that works standalone! I am running it on a remote server and I don’t have access to root to install things like this, so I can no longer use the most up to date version :(

  44. Paul Battley

    Wrote at 2008-05-08 21:58 UTC using Firefox 3.0b5 on Linux:

    Tom, setup.rb accepts a whole bunch of options that will allow you to install the program wherever you like.

    More straightforwardly, though, if you expand the package in your home directory, you can do something like this:

    ruby ~/iplayer-dl/bin/iplayer-dl -I ~/iplayer-dl/lib -- PID

  45. Jonathan Shuttleworth

    Wrote at 2008-05-09 17:45 UTC using Firefox 2.0.0.14 on Windows XP:

    Okay, this is annoying. I’ve installed all the files as per your instructions, but when I try to download a page, it gives me a Permission Denied error.

    ruby: Permission denied—iplayer-dl (LoadError)

    Any ideas?

  46. Jonathan Shuttleworth

    Wrote at 2008-05-09 17:48 UTC using Firefox 2.0.0.14 on Windows XP:

    Sorry, problem seems to be solved. I’ll write back if something else goes wrong.

  47. JJ

    Wrote at 2008-05-10 07:54 UTC using Firefox 2.0.0.14 on Windows XP:

    Any chance of some idiot proof instructions of how to install this?

    I’ve followed the instructions here but get:

    No such file or directory—~/iplayer-dl/bin/iplayer-dl (LoadError)

    I am obviously doing something wrong but I have no idea what.

  48. Tom Davie

    Wrote at 2008-05-10 10:15 UTC using Safari 525.18 on Mac OS X:

    Hi Paul, Thanks for your instructions, I had assumed there was a way of doing this, but given up my research after ruby setup.rb configure—prefix hadn’t worked.

    Anyway, as a couple of other guys have pointed out, the instruction you gave there result in a LoadError:

    iplayer-dl/bin/iplayer-dl:9:in `require’: no such file to load—iplayer (LoadError)

    Any way around this?

    Thanks

    Tom

  49. Stuart McNicholas

    Wrote at 2008-05-12 08:58 UTC using Safari 525.18 on Mac OS X:

    Dear Paul,
    Brilliant program, extremely useful.

    There seems to be a problem with “Child of our Time”.

    tiny:~ stuart$ iplayer-dl http://www.bbc.co.uk/iplayer/page/item/b00b8vm5.shtml

    Resuming download at 0 bytes.
    An MP4 URL was found, but the download failed.

    This is with svn co yesterday (Sunday) evening.

    Are the beeb doing something clever now?

    Stuart

  50. Paul Battley

    Wrote at 2008-05-12 09:10 UTC using Firefox 3.0b5 on Mac OS X:

    Stuart, I’ve seen this a few times. I think it’s just a server misconfiguration: either the MP4 file hasn’t been uploaded to the server, or the server’s out of rotation, or something like that. That’s my working hypothesis.

    If anyone can show me that the programme works on a real iPhone/iPod, I’ll have a closer look.

  51. rak

    Wrote at 2008-05-12 15:46 UTC using Firefox 3.0b5 on Mac OS X:

    Some iPlayer related news… Stephen Fry, known Apple fan and tech junky, made a speech last week on the future of public service broadcasting, in which he mentions the iPlayer, projects like iplayer-dl and, er, his mother – watch the clip here…

  52. Tom Davie

    Wrote at 2008-05-12 19:27 UTC using Safari 525.18 on Mac OS X:

    For those trying to run iplayer-dl from their home directory, here’s how I did it:

    svn co http://paulbattley.googlecode.com/svn/iplayer-dl #Note, this process didn’t work with the tarball above, only the svn version
    ruby setup.rb config
    ruby setup.rb install—prefix=~/
    mv ~/usr/local/bin ../.. # Setting the prefix doesn’t work correctly apparently
    mv ~/usr/local/lib ../..
    export LOAD_PATH=~/lib/ruby/site_ruby/1.8:$LOAD_PATH
    export PATH=~/bin:$PATH
    iplayer-dl “woot, it seems to be working now”

  53. Daniel Martin

    Wrote at 2008-05-14 18:12 UTC using Firefox 2.0.0.14 on Linux:

    Hi,

    I’ve noticed that you haven’t put iplayer-ld under a specific copyright license. This means that technically it’s illegal for people to run it, modify it, distribute it. When you get time would you consider choosing a license for it? Personally I would suggest the GNU GPL the ideals of which can be found at http://www.gnu.org/philosophy/free-sw.html .

    BTW: Thanks for your work on iplayer-dl, I find it very useful!

    dacm

  54. Paul Battley

    Wrote at 2008-05-14 19:32 UTC using Firefox 3.0b5 on Linux:

    Daniel, it’s under MIT. The repository is marked as such, and there’s now a COPYING file in the project directory too. It’s not in the current snapshot, but it will be in the next one.

  55. bosel

    Wrote at 2008-05-14 19:59 UTC using Firefox 2.0.0.14 on Linux:

    latest svn gives:

    /usr/lib64/ruby/1.8/net/http.rb:564:in `initialize’: Connection refused – connect(2) (Errno::ECONNREFUSED)
    from /usr/lib64/ruby/1.8/net/http.rb:564:in `open’
    from /usr/lib64/ruby/1.8/net/http.rb:564:in `connect’
    from /usr/lib64/ruby/1.8/timeout.rb:48:in `timeout’
    from /usr/lib64/ruby/1.8/timeout.rb:76:in `timeout’
    from /usr/lib64/ruby/1.8/net/http.rb:564:in `connect’
    from /usr/lib64/ruby/1.8/net/http.rb:557:in `do_start’
    from /usr/lib64/ruby/1.8/net/http.rb:546:in `start’
    from /usr/lib64/ruby/1.8/net/http.rb:1044:in `request’
    from /usr/lib64/ruby/1.8/net/http.rb:957:in `request_get’
    from /usr/lib64/ruby/site_ruby/1.8/iplayer/browser.rb:50:in `get’
    from /usr/lib64/ruby/site_ruby/1.8/iplayer/downloader.rb:21:in `get’
    from /usr/lib64/ruby/site_ruby/1.8/iplayer/downloader.rb:45:in `download’
    from /root/scripts/converter/iplayer-dl/bin/iplayer-dl:92
    from /root/scripts/converter/iplayer-dl/bin/iplayer-dl:91:in `open’
    from /root/scripts/converter/iplayer-dl/bin/iplayer-dl:91

    any ideas on what’s going on?

  56. AudioBear

    Wrote at 2008-05-15 09:29 UTC using Firefox 3.0b5 on Mac OS X:

    Has the bbc url changed?
    http://www.bbc.co.uk/iplayer/page/item/b00bbybk.shtml?filter=txdate%3A14-05&filter=txslot%3Aevening&start=3&scope=iplayerlast7days&version_pid=b00bby8c

    I don’t remember the pid being different from the ’.shtml’ bit?

  57. Paul Battley

    Wrote at 2008-05-15 10:53 UTC using Firefox 3.0b5 on Mac OS X:

    AudioBear, ignore the version_pid: it works with the regular pid:

    iplayer-dl b00bbybk

  58. Em

    Wrote at 2008-05-16 10:55 UTC using Internet Explorer 6.0 on Windows XP:

    Hi

    Get this message any ideas as I have been using this OK for a while?

    from c:/ruby/lib/ruby/1.8/net/http.rb:564:in `open’
    from c:/ruby/lib/ruby/1.8/net/http.rb:564:in `connect’
    from c:/ruby/lib/ruby/1.8/timeout.rb:48:in `timeout’
    from c:/ruby/lib/ruby/1.8/timeout.rb:76:in `timeout’
    from c:/ruby/lib/ruby/1.8/net/http.rb:564:in `connect’
    from c:/ruby/lib/ruby/1.8/net/http.rb:557:in `do_start’
    from c:/ruby/lib/ruby/1.8/net/http.rb:546:in `start’
    from c:/ruby/lib/ruby/1.8/net/http.rb:1044:in `request’
    from c:/ruby/lib/ruby/1.8/net/http.rb:957:in `request_get’
    from iplayer-dl.rb:54:in `http_get’
    from iplayer-dl.rb:89

  59. AudioBear

    Wrote at 2008-05-16 17:27 UTC using Firefox 3.0b5 on Mac OS X:

    Paul said “AudioBear, ignore the version_pid: it works with the regular pid:
    iplayer-dl b00bbybk”

    Thanks, Paul I got there in the end, does the code parse out the correct PID if you provide a full URL?

  60. MrBean

    Wrote at 2008-05-16 20:01 UTC using Firefox 3.0b5 on Linux:

    A basic question: does this deal with proxies that require username/password pairs? The standard form that gets passed through by other programs is http_proxy=”name:password@proxy.blah.uk:port/” but the program appears not to like it, responding with ”/usr/lib/ruby/1.8/uri/common.rb:436:in `split’: bad URI”. This is after I supply a -p option, otherwise it fails with the missing inject method mentioned by Wynne earlier, which I’m assuming is that it can’t make a valid connection because it isn’t figuring out the proxy.

    In case it matters, this is with svn rev 130, ruby 1.8.6 (2007-09-24 patchlevel 111) [x86_64-linux] on ubuntu.

  61. Paul Battley

    Wrote at 2008-05-16 22:46 UTC using Firefox 3.0b5 on Linux:

    MrBean: It doesn’t support password-protected proxies. I’ve never encountered one, and I don’t know how they work, but I’d welcome a patch.

  62. Henry Bradley

    Wrote at 2008-05-17 18:04 UTC using Safari 525.18 on Mac OS X:

    Is this the BBC now shutting the door? When I try and use iplayer-dl I now get this lot:

    /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:54:in `rbuf_fill’: execution expired (Timeout::Error)
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:56:in `timeout’
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:76:in `timeout’
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/protocol.rb:132:in `rbuf_fill’
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/protocol.rb:116:in `readuntil’
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/protocol.rb:126:in `readline’
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:2029:in `read_status_line’
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:2018:in `read_new’
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:1059:in `request’
    ... 6 levels…
    from /Library/Ruby/Site/1.8/iplayer/downloader.rb:45:in `download’
    from iplayer-dl:90
    from iplayer-dl:89:in `open’
    from iplayer-dl:89

  63. fophillips

    Wrote at 2008-05-17 19:14 UTC using Firefox 3.0 on Linux:

    I can confirm Henry’s problem.

  64. James Raybould

    Wrote at 2008-05-17 20:08 UTC using Safari 525.18 on Mac OS X:

    I’ll third that, I’m using a proxy server but it use to work fine before today

    /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:54:in `rbuf_fill’: execution expired (Timeout::Error)
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:56:in `timeout’
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:76:in `timeout’
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/protocol.rb:132:in `rbuf_fill’
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/protocol.rb:104:in `read_all’
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:2229:in `read_body_0’
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:2182:in `read_body’
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:2207:in `body’
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:2146:in `reading_body’
    ... 7 levels…
    from /Library/Ruby/Site/1.8/iplayer/downloader.rb:77:in `programme_page’
    from /Library/Ruby/Site/1.8/iplayer/downloader.rb:25:in `versions’
    from /Library/Ruby/Site/1.8/iplayer/downloader.rb:32:in `available_versions’
    from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/iplayer-dl:73

  65. Paul Battley

    Wrote at 2008-05-17 20:52 UTC using Firefox 3.0b5 on Linux:

    I can confirm the error. I’m trying to get hold of a header dump from an iPhone session to work out what’s changed.

  66. mjp

    Wrote at 2008-05-17 22:32 UTC using Firefox 2.0.0.14 on Windows Vista:

    I can confirm the above, also the Xbox Media Centre iPlayer script has stopped working today.

    Error log below:

    D:DocumentsToolsiPlayer Downloadiplayer-dl-0.1.2bin>ruby iplayer-dl http://
    www.bbc.co.uk/iplayer/page/item/b00bbpsb.shtml?src=ip_mp
    c:/ruby/lib/ruby/1.8/net/protocol.rb:133:in `sysread’: end of file reached (EOFE
    rror)
    from c:/ruby/lib/ruby/1.8/net/protocol.rb:133:in `rbuf_fill’
    from c:/ruby/lib/ruby/1.8/timeout.rb:56:in `timeout’
    from c:/ruby/lib/ruby/1.8/timeout.rb:76:in `timeout’
    from c:/ruby/lib/ruby/1.8/net/protocol.rb:132:in `rbuf_fill’
    from c:/ruby/lib/ruby/1.8/net/protocol.rb:116:in `readuntil’
    from c:/ruby/lib/ruby/1.8/net/protocol.rb:126:in `readline’
    from c:/ruby/lib/ruby/1.8/net/http.rb:2029:in `read_status_line’
    from c:/ruby/lib/ruby/1.8/net/http.rb:2018:in `read_new’
    ... 7 levels…
    from c:/ruby/lib/ruby/site_ruby/1.8/iplayer/downloader.rb:54:in `downloa
    d’
    from iplayer-dl:90
    from iplayer-dl:89:in `open’
    from iplayer-dl:89

  67. fophillips

    Wrote at 2008-05-17 22:55 UTC using Firefox 3.0 on Linux:

    It appears to be working again now.

  68. mjp

    Wrote at 2008-05-17 23:05 UTC using Firefox 2.0.0.14 on Windows XP:

    I can confirm it is now working, along with the Xbox script as well.

    The iPlayer website frequently 502’d on me today, downloading with the BBC client was very slow.

    Bad day I assume!

  69. Henry Bradley

    Wrote at 2008-05-18 07:44 UTC using Safari 525.18 on Mac OS X:

    Sorted here too. I’d had loads of errors during the day as well so maybe it was just generically shafted.

  70. Paul Battley

    Wrote at 2008-05-18 08:27 UTC using Firefox 3.0b5 on Linux:

    It’s working once again for me, too, so it looks like it was all just a false alarm. Thanks for the updates!

  71. MrBean

    Wrote at 2008-05-18 09:45 UTC using Firefox 3.0b5 on Linux:

    Many thanks for the reply about password based proxies. Unfortunately I don’t know how they work either (or indeed networks in general :-) work ) so no patch is likely from me.

  72. Shaheed

    Wrote at 2008-05-18 15:38 UTC using Firefox 2.0.0.12 on Mac OS X:

    Hey! I can’t seem to download the latest apprentice episode using iplayer-dl. Is there a problem? It keeps saying it is not available in MP4 but i used a user agent switcher and it seems that it is available for iphone…

    The link is http://www.bbc.co.uk/iplayer/page/item/b00bbf8t.shtml?src=ip_mp

  73. Paul Battley

    Wrote at 2008-05-18 20:42 UTC using Firefox 3.0b5 on Linux:

    Shaheed, it just hasn’t been converted to MP4 yet. Give it a day or so.

  74. Shaheed

    Wrote at 2008-05-19 16:31 UTC using Firefox 2.0.0.13 on Mac OS X:

    Paul, are you sure? Because Apprentice was screened 5 days ago…usually the max it has taken for me to wait for mp4 is a couple of days…

  75. Upekshapriya

    Wrote at 2008-05-19 23:51 UTC using Firefox 2.0.0.14 on Windows XP:

    It’d be cool if there were a Windows GUI (OK I know!) – it would make it even easier to get BBC programmes onto my iPod :-)

  76. Digital

    Wrote at 2008-05-20 23:13 UTC using Safari 525.18 on Mac OS X:

    I can’t get child of our time to download, have tried both 1st & second episodes but just get the message “An MP4 URL was found, but the download failed.”

    Example iplayer-dl b00bf6lt

  77. tom kelly

    Wrote at 2008-05-22 14:48 UTC using Internet Explorer 6.0 on Windows 2000:

    ok jailbroken iphone have the ability to install a limited version of ruby, would it be possable to compile a stand alone version on this app?

  78. Andrew L

    Wrote at 2008-05-23 23:54 UTC using Firefox 2.0.0.14 on Windows XP:

    Hello,

    Thank-you very much for all your work, after finding your posts I have managed to piece together what you were talking about from having little scripting knowledge.

    I have managed to use Ruby to get your last script working and tested it out today on ‘Women in Black’

    http://www.bbc.co.uk/iplayer/page/item/b00bgnjk.shtml

    and was quite chuffed when I managed to get it to work.

    But I am trying to download a BBC iPlayer program not hosted on their pages,

    http://pendulum.com/

    There is currently a video of their full ‘Radio 1’s Big Weekend’ set. But your script understandably won’t work on this page. I cant find a PID for this program, but I have found this xml file for it

    http://www.bbc.co.uk/radio1/bigweekend/2008/emp/xml/pendulum_1.xml

    Wondering if anybody could give me any help on downloading this video.

    I will write up a tutorial for windows users over the next few weeks to help everyone who is reading your updates and scratching their heads

  79. charles

    Wrote at 2008-05-28 09:29 UTC using Firefox 2.0.0.14 on Windows XP:

    I guess BBC has changed the iPlayer again. I get this message:

    Unable to parse the programme page. Possible reasons:
    1. The iPlayer has changed
    2. You are outside the UK (and not using a UK proxy).

    I live in London, therefore it excludes option 2. (my internet provider is actually based in London, and I do not use any proxies…)

  80. Paul Battley

    Wrote at 2008-05-28 10:22 UTC using Firefox 3.0 on Mac OS X:

    Charles, I can still download, at least on some programmes. I don’t think they’ve changed anything.

    There’s another possibility: you are in the UK, but the BBC IP geolocation thinks that you aren’t.

    Which programme was it? Which PID?

  81. adapa

    Wrote at 2008-05-31 10:56 UTC using Firefox 2.0.0.14 on Windows Server 2003:

    When I execute the script with any options, all I get is:

    c:/ruby/bin/iplayer-dl.rb:125: syntax error, unexpected kRESCUE, expecting kEND
    rescue ParsingError
    ^
    c:/ruby/bin/iplayer-dl.rb:132: syntax error, unexpected kRESCUE, expecting kEND
    rescue FileUnavailable
    ^
    c:/ruby/bin/iplayer-dl.rb:137: syntax error, unexpected kRESCUE, expecting kEND
    rescue MP4Unavailable
    ^
    c:/ruby/bin/iplayer-dl.rb:142: syntax error, unexpected kRESCUE, expecting kEND
    rescue MetadataError
    ^
    c:/ruby/bin/iplayer-dl.rb:148: syntax error, unexpected $end, expecting kEND

    Any ideas ?

  82. Paul Battley

    Wrote at 2008-06-01 10:37 UTC using Firefox 3.0b5 on Linux:

    Adapa, are you using the snapshot, or a Subversion checkout?

  83. Dan Cruikshank

    Wrote at 2008-06-01 20:45 UTC using iPhone on Mac OS X:

    Hi,
    I’m trying to d/l
    http://www.bbc.co.uk/iplayer/page/item/b00brpqp.shtml
    using iplayer-dl-0.1.3 on Linux. I’m getting the “Signed” version.
    How can I get the “Original” version?

    Thanks,
    Dan

  84. Paul Battley

    Wrote at 2008-06-01 21:38 UTC using Firefox 3.0b5 on Linux:

    Dan: the short answer is ‘go back in time’.

    You’re too late, I’m afraid. The signed versions are usually broadcast/uploaded later than the original versions. In this case, the original version was only available for download until 28th May, but the signed version is still available.

    There’s nothing I can do to help!

  85. Daniel Hardy

    Wrote at 2008-06-05 18:39 UTC using Firefox 3.0 on Mac OS X:

    Paul,

    Three times i’ve downloaded this – >> (http://www.bbc.co.uk/iplayer/page/item/b00byd29.shtml?src=ip_mp)

    Once with the latest version of Beeb Downloader, then with a previous version of your script, then again with the latest version of the script.

    Each time, the program seemed to download fine, but when I try to play the video, I get various error messages, Quicktime says “The Movie could not be opened, ... The file is not a movie file” or media players give me similar messages.

    Had similar experiences with other programs.

  86. Bob

    Wrote at 2008-06-06 09:10 UTC using Safari 525.20 on Mac OS X:

    Ditto. I have downloaded five different programmes in the last day or so and all resulted in the message “The Movie could not be opened, ... The file is not a movie file” both in Quicktime and VLC. They cannot be imported into iTunes.

  87. Paul Battley

    Wrote at 2008-06-06 13:12 UTC using Firefox 3.0 on Mac OS X:

    The fact that the files are downloading and the programmes do work on the iPhone make me suspect that they’ve found a way to construct QuickTime files that only the iPhone can play.

    The next step is to try to play a downloaded file on a (jailbroken) iPhone or iPod Touch.

  88. LSF

    Wrote at 2008-06-06 14:29 UTC using Firefox 2.0.0.14 on Mac OS X:

    Yeah, I have the same problem. My ipod touch is jailbroken but I’m not sure how to copy a downloaded file across. Itunes normally does that but as has been mentioned itunes won’t import these files. Could attempt to copy the files over via ssh but doubt that would work as it seems the itunes database on the ipod would also need to be updated.

  89. Daniel Hardy

    Wrote at 2008-06-06 21:20 UTC using Firefox 3.0 on Mac OS X:

    It’ll be hard to move one of these files over to an iphone or ipod touch since itunes won’t import the files.

  90. Paul Battley

    Wrote at 2008-06-06 21:48 UTC using Firefox 3.0b5 on Linux:

    I realise that iTunes hates these files, but can’t you just embed it in a web page, iPlayer-style?

  91. LSF

    Wrote at 2008-06-06 22:35 UTC using Firefox 2.0.0.14 on Mac OS X:

    Great minds think alike – I just uploaded the movie to my web server and found that the ipod touch couldn’t play it either.

  92. MarC

    Wrote at 2008-06-08 01:26 UTC using Opera 9.50 on Linux:

    Same thing here. The .mov’s download, but mplayer / gmplayer report that the files have a broken header.

    Seems like some sort of encryption is being applied here

  93. wm.wragg

    Wrote at 2008-06-09 16:12 UTC using Firefox 3.0b5 on Linux:

    Someone has worked out how to decript the downloaded files (from http://beebhack.wikia.com/wiki/Beebhack#iPlayer_TV):

    Update to video file encoding as of 8th June 2008

    In fact the DRM appears to be a simple XOR of a large section of the plain vanilla MP4 with a repeating two-byte pattern. Specifically, the first 0×2800 bytes are unchanged, then the next bytes are XOR’d with values beginning 0×3c, 0×53, 0×3c, etc., then the last 0×400 bytes are unchanged. Just to make things interesting the two bytes immediately before the last 0×400 clean ones are XOR’d with reversed values. So (for example) the file is XOR’d with 0, 0,..(0×2800 times).., 0×3c, 0×53, .... ,0×3c, 0×53, 0×3c, 0×53, 0×53, 0×3c, 0, 0, ..(0×400 times)... To recover the original file, just XOR again with these values.

    See http://linuxcentre.net/iplayer_decode for the perl script.

  94. MarC

    Wrote at 2008-06-09 18:09 UTC using Opera 9.50 on Linux:

    I can confirm the perl script decoder works. After downloading the iplayer content and ususal, and then decoding it with http://linuxcentre.net/iplayer_decode I can then view the content with gmplayer / mplayer as before.

    Thanks :)

  95. Paul Battley

    Wrote at 2008-06-12 13:42 UTC using Firefox 3.0 on Mac OS X:

    Comments are now closed: please visit my new project page instead.