Recent Updates Page 2 Toggle Comment Threads | Keyboard Shortcuts

  • raj 6:46 pm on October 14, 2013 Permalink | Reply
    Tags: , iceland   

    Iceland: Where one in 10 people will publish a book –

  • raj 6:24 pm on September 5, 2013 Permalink | Reply
    Tags: dvd,   

    A robot for digitizing DVDs:

  • raj 7:51 pm on September 4, 2013 Permalink | Reply
    Tags: cnc,   

    MatrixShotâ„¢ 1.0 First Run

  • raj 11:32 pm on August 8, 2013 Permalink | Reply

    A python interface to 

    Jake pointed out today that he wrote a cool python wrapper for that made it easy to upload files. I added downloading support and a few other features:

    #The Internet Archive stores data in "items". You can query the archive using an item identifier:
    >>> import archive
    >>> item = archive.Item('stairs')
    >>> print item.metadata
    #Items contains files, which can be downloaded:
    >>> f = item.file('glogo.png')
    >>> #writes to disk
    #You can iterate over files:
    >>> for f in item.files():
    ...     print, f.sha1
    #You can use the IA's S3-like interface to upload files to an item.
    #You need to supply your IAS3 credentials in environment variables in order to upload.
    #You can retrieve S3 keys from
    >>> import os;
    >>> os.environ['AWS_ACCESS_KEY_ID']='x'; os.environ['AWS_SECRET_ACCESS_KEY']='y'
    >>> item.upload('myfile')
  • raj 6:01 pm on July 17, 2013 Permalink | Reply
    Tags: canon, chdk, magic lantern   

    Magic Lantern is third-party firmware (like CHDK) that works on Canon DSLRs:

  • raj 11:44 pm on June 26, 2013 Permalink | Reply
    Tags: , uploader   

    Plotting uploads/day 

    We are beta testing a HTML5 Uploader, and I wanted to see how much use it was getting. Here is how I plotted the usage. First, install ipython html notebook, then install matplotlib 1.3rc2 to use the xkcd style, then start iPython notebook:

    $ source ~/pyenvs/notebook/bin/activate
    $ ipython notebook --profile=nbserver

    Then, in iPython notebook, use the advanced search engine to see how many uploads per day we were getting:

    import json
    import urllib
    import datetime as dt
    urlbase = ''
    date = dt.datetime.utcnow()
    for i in range(90):
        date -= dt.timedelta(days=1)
        params = {'q':'scanner:"Internet Archive HTML5 Uploader" AND publicdate:'+date.strftime('%Y-%m-%d'),
        url = urlbase % urllib.urlencode(params)
        f = urllib.urlopen(url)
        o = json.load(f)

    Now that I have the data, I can plot it, using xkcd-style:

    #Turn on XKCD style
    plot(x, y)
    plt.title("HTML Uploader Items/Day")
    #Save PNG
    fig = plt.gcf()

    It looks like this:

  • raj 10:22 pm on June 24, 2013 Permalink | Reply
    Tags: gibson

  • raj 11:04 pm on June 19, 2013 Permalink | Reply
    Tags: , , xkcd   

    I wanted to update the virtualenv I use… 

    I wanted to update the virtualenv I use for iPython HTML Notebook with the latest matplotlib, which includes XKCD-style plotting. I first set up the virtualenv using these instructions, and then used pip to install the RC2 version of matplotlib using the tarball (since 1.3.0rc2 is not in pypi):

    $ pip install
    #pip installs matplotlib from tarball, but half-uninstalls the dateutil package that we previously installed
    $ pip uninstall python-dateutil
    $ pip install python-dateutil

    Now, we need to install the Humor Sans font. I downloaded the ttf and put it in ~/.fonts/Humor-Sans.ttf

    After installing the font, we need to make sure the fontList.cache file is rebuilt (Thanks Georg!):

    $ rm ~/.matplotlib/fontList.cache

    We should now be able to produce xkcd-style plots just by running this command in our notebook before plotting:


    However, there are a couple errors with matplotlib 1.3.0rc2:

    • matplotlib.font_manager’s findfont() can’t find Humor-Sans.ttf, which I placed in matplotlib.font_manager.path, so manaully set the fonts for now
    • The XKCD artist style doesn’t work with plot_time() very well, so handle x-axis dates manaully as well

    I downloaded some data from graphite using &rawData=true and pasted that into my ipython notebook. This is sample graph of what it looks like:
    matplotlib xkcd

    Here is the code I used:

    Update: The code below contains workarounds for a font issue that is now fixed (Thanks Georg!). See this post for a better example

    #Use the XKCD style
    #Manually set the font, since I can't seem to get matplotlib to load Humor Sans automatically
    ax = pylab.axes()
    prop = matplotlib.font_manager.FontProperties(fname=matplotlib.font_manager.path+'/Humor-Sans.ttf', size=16)
    #Set the Y axis
    def fmt_mb(x, pos):
        if x > 14e6:
            return ""
        return '%0.0fMB/min' % (x*1e-6)
    yfmt = FuncFormatter(fmt_mb)
    #Set the X axis
    start = 1371584040
    end = 1371670440
    timestamps = numpy.linspace(start, end, len(x))
    dates=[datetime.datetime.utcfromtimestamp(ts) for ts in timestamps]
    xfmt = DateFormatter('%m-%d %H:%M')
    #Set the title, and manually specify font due to matplotlib findfont issue
    plt.title("HTML5 Uploader MB/min for the last 24 hours", fontproperties=prop) 
    #Plot the moving average
    values = matplotlib.mlab.movavg(x, 15)
    ax.plot(dates[:len(values)], values)
    #Manually set font for the labels, due to findfont issue
    for label in ax.get_xticklabels():
    for label in ax.get_yticklabels():
    #Save PNG
    fig = matplotlib.pyplot.gcf()
    • Georg Nebehay 1:03 pm on July 8, 2013 Permalink | Reply

      I ran into the same issues with matplotlib not recognising Humor Sans but this thread here
      provided me with a solution. After installing a new font it is necessary to regenerate the matplotlib font list cache, which can be done by deleting ~/.matplotlib/fontList.cache on Unix systems. I also had to rename the file Humor_Sans.ttf to Humor Sans.ttf, and now it works like a charm.

      • raj 11:30 pm on July 17, 2013 Permalink | Reply

        Thanks, Georg! Removing ~/.matplotlib/fontList.cache worked for me, without having to rename my font file, which on my system is ~/.fonts/Humor-Sans.ttf

        I’ll update the instructions above.

  • raj 6:40 pm on May 9, 2013 Permalink | Reply
    Tags: documentary   

    Documentary about the Internet Archive 

    By Jonathan Minard, John Behrens, Alexander Porter, and Fearghal O’dea

    Internet Archive from Deepspeed media on Vimeo.

  • raj 6:02 pm on January 17, 2013 Permalink | Reply
    Tags: aaronsw   

    Memorial for Aaron 

    There will be a memorial for Aaron at the Internet Archive on Thrusday, January 24.

    The flag at the Internet Archive is flying at half mast in memory of Aaron:

Compose new post
Next post/Next comment
Previous post/Previous comment
Show/Hide comments
Go to top
Go to login
Show/Hide help
shift + esc