Python / SOAP: A First Encounter
Thursday, February 9, 2006 by Scott KarlinThis entry describes my recent foray into creating a Simple Object Access Protocol (SOAP) client. Perhaps some will find the information useful and others can provide some helpful hints in the comments.
The web service I wanted to access exports a method that returns an array of strings each with a particular structure (this will be important later). The details of this service are not important so let’s call the method getList. Since this client would be run from a Linux command line and I wanted to brush-up on my Python programming skills, I went to work looking for an appropriate SOAP library for Python. A search on Google led me to two primary choices, SOAPpy and ZSI, both part of the Python Web Services project. SOAPpy looked to be the easier to use and so I installed SOAPpy version 0.12.0 and started with this bit of code:
from SOAPpy import WSDL
server = WSDL.Proxy("http://blah.blah.blah/blah?wsdl")
server.soapproxy.config.dumpSOAPOut = 1
server.soapproxy.config.dumpSOAPIn = 1
result = server.getList("param1", "param2")
# Process result...
From everything I read, this should have worked. Unfortunately, I was greeted with a “duplicate attribute” exception from the depths of the xml.sax parser:
Traceback (most recent call last):
File "soaptest.py", line 7, in ?
result = server.getList("param1", "param2")
File "/usr/lib/python2.4/site-packages/SOAPpy/Client.py",
line 470, in __call__
return self.__r_call(*args, **kw)
File "/usr/lib/python2.4/site-packages/SOAPpy/Client.py",
line 492, in __r_call
self.__hd, self.__ma)
File "/usr/lib/python2.4/site-packages/SOAPpy/Client.py",
line 395, in __call
p, attrs = parseSOAPRPC(r, attrs = 1)
File "/usr/lib/python2.4/site-packages/SOAPpy/Parser.py",
line 1049, in parseSOAPRPC
t = _parseSOAP(xml_str, rules = rules)
File "/usr/lib/python2.4/site-packages/SOAPpy/Parser.py",
line 1032, in _parseSOAP
raise e
xml.sax._exceptions.SAXParseException: :1:514: duplicate attribute
Armed with this traceback and the fact that the WDSL file (when viewed with a browser) indicated that the server is “Apache Axis version 1.2.1,” I searched the web and found these bug reports:
With the SOAP debugging turned on, I could see that the server was returning the expected list of items—the XML parser was simply preventing me from getting at them. As it turned out, there was indeed a duplicated xsi:type="soapenc:Array" attribute in the returned SOAP message. The bug reports above imply that this issue is fixed in Axis 1.3; however, since the particular Axis SOAP servlet I needed to use was part of a bundled system (that I didn’t have control over), upgrading Axis was not an option.
The Workaround
The workaround (actually a “hack”) that I used was to temporarily redirect the SOAP debugging output sent to stdout to an internal buffer. From there, I could leverage the fact that the returned strings had a nice structure to extract them from the buffer using a regular expression:
from SOAPpy import WSDL
import xml.sax
import sys
import re
class Sniff:
def __init__(self):
self.b = ""
def write(self, s):
self.b = self.b + s
def flush(self):
pass
def buffer(self):
return self.b
server = WSDL.Proxy("http://blah.blah.blah/blah?wsdl")
server.soapproxy.config.dumpSOAPOut = 1
server.soapproxy.config.dumpSOAPIn = 1
sniff = Sniff()
sys.stdout = sniff
try:
result = server.getList("param1", "param2")
except xml.sax.SAXParseException, e:
pass
sys.stdout = sys.__stdout__
c = re.compile(r">([a-z0-9]+):([^,:< ]+),([^:<]+):([^<]+)<")
result = re.findall(c, sniff.buffer())
# Process result...
I should mention that before I resorted to this hack, I did try using ZSI and even cSOAP (a C library for SOAP). ZSI gave a richer set of controls over the SOAP protocol; however, I found that with so many knobs to fiddle with, I could only elicit from the server a 500 Internal Server Error response at the HTTP layer with a corresponding Server.userException response at the SOAP layer. Even if I could get the outgoing SOAP message to be accepted by the server, I wasn’t sure if I wouldn’t have the same parsing problem as I did with the SOAPpy implementation. With cSOAP, I went straight to my C programming roots. Unfortunately, the installation documentation was a little sparse and I didn’t pursue this very far before the above hack came to mind.
Of course, I would rather not have had to resort to the workaround. My current implementation is more brittle than it should be. For a web service this simple, I would have preferred XML-RPC.
