Making The New Amazon Product API Easy to Work With

I'm working on a new startup idea that I think will really rock the niche it's in and have been using the Amazon Product API a bunch.  They updated the API recently and now require you to send your associate id along with the query.

I've tried several xml parsing libraries and several Amazon libraries and have decided that the current combination of BeautifulSoup and bottlenose are what I enjoy working with the most.  I've been able to bang out tools for myself using this combo very quickly and have actually had fun building each one of them for a change.

import bottlenose
from BeautifulSoup import BeautifulSoup

# Replace these values with your correct info
AWS_KEY = 'key_goes_here'
SECRET_KEY = 'secret_goes_here'
ASSOCIATE_ID = 'your_associate_id_goes_here'

# Create a bottlenose object and initialize with our credentials
amazon = bottlenose.Amazon(AWS_KEY, SECRET_KEY)

# Query Amazon's Product API to perform an item lookup via the ASIN, returns the large response # group containing a ton of info, note that we must send the associate id now
response = amazon.ItemLookup(ItemId="B0018AFK38", ResponseGroup="Large", AssociateTag=ASSOCIATE_ID)

# Create a BS object from the XML response
soup = BeautifulSoup(response)

# A sample of the data that can be retrieved
print soup.find('itemid').string
print soup.find('formattedprice').string
print soup.find('detailpageurl').string
print soup.find('sku').string
print soup.find('content').string
print soup.find('title').string
print soup.find('department').string
print soup.find('smallimage').url.string
print soup.find('mediumimage').url.string
print soup.find('largeimage').url.string

My Five Favorite Python Library Finds on Github This Week

1. https://github.com/edsu/microdata - (python library for extracting html5 microdata)
   
   I've been working with HTML5 a lot especially when dealing with Wordpress blogs and was interested in what's available for HTML5 in Python.
   


2. https://github.com/pneff/wsgiservice - (Python WSGI framework to create REST web services)
   
   Over the last few weeks I've had the (mis)fortune of having to develop a public API for a company using PHP and the Slim framework.  Slim isn't bad but PHP is getting long in the teeth and it's just not nearly as fun to develop in as Python and Ruby.  wsgiservice looks like a good choice (from out of many) for building RESTful APIs.


3. https://github.com/mcfunley/etsy-python - (Python access to the Etsy API)
   
   Esty's awesome.  Python's awesome.  Putting the two together and making a cool site or tool of some sort seems like a good idea and this module can certainly help.
   


4. https://github.com/liamks/pyitunes - (Python Itunes Library parser)
   
   iTunes is hated my many but I love it and this library (although not maintained as well as it could be) allows access to my music collection.  I use it for fixing meta tags on my audio files.
   


5. https://github.com/solex/python-odesk - (Python bindings for oDesk API )
   
   I work full-time on oDesk these days as a developer and am building some custom tools to make finding new jobs easier and to make visualizing some of my key statistics easier.  For instance, oDesk doesn't give you an earnings to date readout and I would like that.  I would also like to be able to filter jobs out so that I'm never shown jobs below my minimum price point.

Working with YouTube API from Python - A Primer

Grabbing data from YouTube with Python is really simple and you can build a ton of apps using this information. It's nothing more than making an HTTP GET request to the proper gdata URL. Then we just take the response in JSON format, decode it, and then can work with it like any other Python dictionary.

#!/usr/bin/env python
import requests
import json

resp = requests.get("http://gdata.youtube.com/feeds/api/standardfeeds/most_popular?v=2&alt=jsonc")
if(resp.status_code == 200):
	data = json.loads(resp.content)
	for item in data['data']['items']:
		print "Video Title: %s" % (item['title'])
		print "Video Category: %s" % (item['category'])
		print "Video ID: %s" % (item['id'])
		print "Video Rating: %f" % (item['rating'])
		print "Embed URL: %s" % (item['player']['default'])
		print
else:
	print "There was a problem retrieving the data"

The results look something like this:

Video Title: Minecraft - Dwarf Hole (Diggy Diggy Hole) Fan Song and Animation
Video Category: Shows
Video ID: fR7EAdPUqvQ
Video Rating: 4.862668
Embed URL: http://www.youtube.com/watch?v=fR7EAdPUqvQ&feature=youtube_gdata_player

Video Title: FAT KID FALLS
Video Category: Entertainment
Video ID: dCd8ojAdUbA
Video Rating: 4.671462
Embed URL: http://www.youtube.com/watch?v=dCd8ojAdUbA&feature=youtube_gdata_player

Video Title: Incredible Shade Illusion!
Video Category: Tech
Video ID: z9Sen1HTu5o
Video Rating: 4.793279
Embed URL: http://www.youtube.com/watch?v=z9Sen1HTu5o&feature=youtube_gdata_player

Video Title: David Guetta feat. Sia - Titanium
Video Category: Music
Video ID: Mb6mS6Yj_UA
Video Rating: 4.845408
Embed URL: http://www.youtube.com/watch?v=Mb6mS6Yj_UA&feature=youtube_gdata_player

Video Title: 50 Cal Machine Gun VS. 250 Watermelons
Video Category: Entertainment
Video ID: XyoAP10uKTk
Video Rating: 4.921269
Embed URL: http://www.youtube.com/watch?v=XyoAP10uKTk&feature=youtube_gdata_player

[ ... cut for space ... ]

Requests - Easiest, Most Enjoyable Way to Make HTTP Requests in Python

I've been working for the last week designing and implementing an API for in-house use at a company in Georgia. The API is being built with PHP5 and a micro-framework called Slim. I would have preferred to build it in web.py but because ultimately this decision was left to the owners who are responsible for maintaining it's being done in PHP (a language that they're familiar with).

One of the sore spots has been testing the API as we put it together. In the end it will be used by the developers that they hired to write mobile phone applications. I settled on using a Python script for testing the API but quickly tired of writing the HTTP request code using httplib2 and urllib2. I set out for a better HTTP Request library and quickly found Requests; an elegant and dead-simple library that makes writing code to work with HTTP Requests actually enjoyable.

>>> import requests
>>> r = requests.get("http://gdata.youtube.com/feeds/api/standardfeeds/most_popular?v=2&alt=jsonc")
>>> r.status_code
200
>>> print r.content
{"apiVersion":"2.1","data":{"updated":"2011-09-09T19:54:17.000-07:00","totalItems":99,"startIndex":1,"itemsPerPage":25,"items":[{"id":"X9YMU0WeBwU","uploaded":"2011-08-17T00:52:31.000Z","updated":"2011-09-11T01:09:51.000Z","uploader":"LadyGagaVEVO","category":"Music","title":"Lady Gaga - Yoü And I (Official Video)","description":"Music video by Lady Gaga performing Yoü And I (Official Video). Lady Gaga's \"Yoü And I\" video received 1,054,214 video views on YouTube prior to the official VEVO video release.  © 2011 Interscope Records","tags":["You","and","new","video","song","official","Laurieann","Gibson","haus","of","gaga","Mother","Monster"],"thumbnail":{"sqDefault":"http://i.ytimg.com/vi/X9YMU0WeBwU/default.jpg","hqDefault

[ ... cut for space ... ]

I think I will also go back through and rewrite a bunch of my YQL code to use Requests instead for it's HTTP request needs. This should be a standard part of your Python toolbox for sure.

Make Your Life as a PHP Developer Twice as Easy With phpsh

I know that this blog is supposed to be mainly about Python but this was just something that I felt needed to be shared regardless. Many of us Python programmers either programmed in PHP at one time or we still do on presently. I use a combination of Ruby, Python, and PHP in my daily work depending on what my clients existing codebase looks like and expect that many of you are no different.

One of the features that I love the most about Python is it's powerful REPL. I love that I can pop over into the console and start testing/fleshing out an idea on the spot. I love being able to test new libraries this way as well which saves me tons of time.

The developers at Facebook have brought PHP developers a powerful REPL now ala-Python to round out a solid toolbox that PHP developers already have. The project is called phpsh and is written in Python.

To give it a go, just clone their git repo to your machine and install the tool via setuptools.

$ git clone git://github.com/facebook/phpsh.git
$ cd phpsh
$ sudo python setup.py install
$ phpsh
Starting php
Install pcntl to enable forking on every command.
type 'h' or 'help' to see instructions & features
php>

Every PHP developer out there that's not aware of this tool is seriously wasting hours upon hours of their time with each project they complete without phpsh. Clone the git repo right now and start enjoying the benefits of an REPL like Python and Ruby developers have enjoyed for the longest time.

Never Forget Your Home IP Address Again!! Python Saves The Day

I have a Linux workstation located at a friend's house that I need to use throughout the day via SSH but I kept forgetting the IP address. He has a dynamic IP address and I didn't feel the need to use a service like dyndns so I wrote a simple Python script that sends a text message to my cell phone each morning at 5:00am. It's quite simple and works great. I hope you find it useful.

#!/usr/bin/env python
import httplib2
import smtplib

h = httplib2.Http(".cache")
res, content = h.request("http://www.whatismyip.com/automation/n09230945.asp", "GET")

s = smtplib.SMTP('smtp.gmail.com:587')
s.starttls()
s.login("username@gmail.com", "password")
s.sendmail("Me", "6025551212\@tmomail.net", content)
s.quit()

Just replace the username, password, and destination email-to-sms gateway for your cell phone carrier.

Easily Interact with the Skype API using Skype4Py

I have been working for a while on automating things that I do using Skype and Applescript. The primary task that I wish to automate is automatically dialing numbers from a call list one at a time and then dialing the next when I click a button on a simple graphical interface. So far I have a working Applescript version but when I came across a python library that was easier to use I thought I would give it a try.

To install Skype4Py just use setup tools and easy_install:

$ sudo easy_install Skype4Py

Here's a quick snippet that displays the users from your friend list that currently have a status that is not included in the badStatusCodes list. It's not much and doesn't perform any error handling but it shows just how easy it is to get started using this library.

import Skype4Py

s = Skype4Py.Skype()
s.Attach()
badStatusCodes = ['UNKNOWN', 'OFFLINE', 'DND', 'LOGGEDOUT']
availFriends = []
for f in s.Friends:
 if not f.OnlineStatus in badStatusCodes:
  availFriends.append(f)
  print "%s, %s" % (f.FullName, f.OnlineStatus)

It seems like quite a useful tool if you are into IP telephony or just want to screw around with creating Skype bots and the like.

Unfortunately Skype4Py doesn't work 100% on OS X because of an issue with threading (and I'm a mac user). For example, if I try to place a call using s.PlaceCall(number) it says that the thread can only be started one time

>>> s.PlaceCall() 
Traceback (most recent call last):
  File "", line 1, in 
  File "build/bdist.macosx-10.3-fat/egg/Skype4Py/skype.py", line 691, in PlaceCall
  File "build/bdist.macosx-10.3-fat/egg/Skype4Py/skype.py", line 877, in _GetActiveCalls
  File "build/bdist.macosx-10.3-fat/egg/Skype4Py/skype.py", line 339, in _Search
  File "build/bdist.macosx-10.3-fat/egg/Skype4Py/skype.py", line 276, in _DoCommand
  File "build/bdist.macosx-10.3-fat/egg/Skype4Py/skype.py", line 778, in SendCommand
  File "build/bdist.macosx-10.3-fat/egg/Skype4Py/api/darwin.py", line 374, in send_command
  File "build/bdist.macosx-10.3-fat/egg/Skype4Py/api/darwin.py", line 327, in attach
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 467, in start
    raise RuntimeError("threads can only be started once")
RuntimeError: threads can only be started once

The following resources are very helpful when working with Skype4Py:

• Skype4Py Online Documentation
• Skype Developer Program

New Python + YQL Scripts Posted On DZONE Snippets

I had mentioned the other day that I would post some of my YQL code for your viewing pleasure and to help you get started with the YQL library; well here it is!

• Scrape free email providers from fepg.net
• Scrape government contract details from transparency.ca.gov
• Subscribe to my Dzone Snippets RSS Feed

I'll Be Adopting Django for My Web Development Projects

From all of the web development frameworks that I have evaluated so far I have liked the Catalyst framework (Perl) the best for my needs; however, it's been murderously difficult to get it working properly on my aging Powerbook G4 running OS X Tiger 10.4.11, and so I've decided that awesome Catalyst-only features that make my day notwithstanding that I should switch to django which was a cinch to get installed and configured.

I'm working today on building a better directory script (along the lines of DMOZ) that I can use for a few directory sites that I want to launch.  I plan to open-source the directory software once complete via Google Code and monetize it by selling templates for it.

We'll see how it goes!