Skip to the Main Content

Note:These pages make extensive use of the latest XHTML and CSS Standards. They ought to look great in any standards-compliant modern browser. Unfortunately, they will probably look horrible in older browsers, like Netscape 4.x and IE 4.x. Moreover, many posts use MathML, which is, currently only supported in Mozilla. My best suggestion (and you will thank me when surfing an ever-increasing number of sites on the web which have been crafted to use the new standards) is to upgrade to the latest version of your browser. If that's not possible, consider moving to the Standards-compliant and open-source Mozilla browser.

December 7, 2004

Ill-formed NOAA

A recent Wired article to the effect that the National Weather Service is now offering their data in XML format got me intrigued enough to want to test out the service.

So I slapped together a little program to fetch the 3-day forecast for Austin

#!/usr/bin/perl

use SOAP::Lite;
use DateTime;

my $lattitude = 30.19;
my $longitude = -97.46;
my $numDays = 3;            # up to 7 days
my $format = '12 Hourly';   # '12 hourly' or '24 hourly'
my $xml_file = '/Users/distler/Sites/blog/files/forecast.xml';

my $startDate = DateTime->now->ymd;

my $serviceURI = 'http://www.nws.noaa.gov/forecasts/xml';
my $method = 'NDFDgenByDay';
my $endpoint = "$serviceURI/SOAP_server/ndfdXMLserver.php";
my $soapAction = "$serviceURI/DWMLgen/wsdl/ndfdXML.wsdl#$method";

my $weather = SOAP::Lite->new(uri => $soapAction,
                            proxy => $endpoint);
my $response = $weather->call(
   SOAP::Data->name($method)
     => SOAP::Data->type(decimal => $lattitude)->name('latitude'),
     => SOAP::Data->type(decimal => $longitude)->name('longitude'),
     => SOAP::Data->type(date    => $startDate)->name('startDate'),
     => SOAP::Data->type(integer => $numDays  )->name('numDays'),
     => SOAP::Data->type(string  => $format   )->name('format')
   );

if ($response->fault) {
     printf "A fault (%s) occurred: %s\n",
     $response->faultcode, $response->faultstring;
} else {
     open(OUT, ">$xml_file");
     print OUT $response->result;
}

The result? Alas … ill-formed XML.

Perhaps I’m a bozo (someone who knows something about SOAP could probably explain where I screwed up above), or perhaps there’s some lesson to be learned here.

Replacing the third line above with

use SOAP::Lite +trace;

you discover that the entire “XML” response is an entity-encoded string in a SOAP wrapper. [Thanks to Sam Ruby for pointing out the “depths of their depravity.”] If you poke around the sample implementation and the referenced PHP include files (which presumably power their server implementation as well), you discover the reason why. All the “XML” is generated by manipulating strings in PHP.

I don’t think Weather.com has anything to fear.

Update (12/8/2004):

I guess I am a bozo, after all. Changing line 9, above, to

my $format = '12 hourly';   # '12 hourly' or '24 hourly'

(with a lowercase ‘h’) produces well-formed XML.

The WSDL for this service, alas, gives no hint that I was doing it wrong. The closest it comes to guiding us on what the correct input should be is:
<operation name="NDFDgenByDay">
   <documentation>
      Returns National Weather Service digital weather
      forecast data. Supports latitudes and longitudes
      for the Continental United States only. Allowable
      values for the input variable "format" are "24 Hourly"
      and "12 Hourly". The input variable "days" is a
      boolean array representing the number of days from
      the current day; elements set to true indicate days
      for which data is requested. For an array element
      is set to "false", data for that day is not to be
      returned.
  </documentation>
<input message="tns:NDFDgenByDayRequest"/>
<output message="tns:NDFDgenByDayResponse"/>
</operation>

which would lead a human to think my original program was delivering what they were expecting.

I still think we learn some valuable lessons here

  1. Well-formed, valid, input should produce well-formed output. The format parameter is declared to be of type xsd:string. If the string I send you isn’t one of the ones you were expecting, send me an error-response. Don’t let your application blow up in my face.
  2. As Sam Ruby notes below, it would be better to send the response as XML, rather than as an escaped string.

Now, about that forecast data …

Update (1/27/2005):

All’s well that end’s well.
Posted by distler at December 7, 2004 6:49 AM

TrackBack URL for this Entry:   http://golem.ph.utexas.edu/cgi-bin/MT-3.0/dxy-tb.fcgi/482

5 Comments & 1 Trackback

Read the post Ill-formed NOAA
Weblog: Sam Ruby
Excerpt: Double encoded, ill-formed weather forecasts, via POST. For those who don't have SOAP::Lite installed, here's a LWP version.
Tracked: December 7, 2004 2:05 PM

Re: Ill-formed NOAA

Thanks for pointing out the problem… we’ll take a look at it. The NDFD-XML service is experimental (regardless of what Wired says) and we are learning everyday. The NDFDgenByDay method was just put up last week. Thanks for helping us to debug it! :-)

Bob Bunge
National Weather Service

Posted by: Bob on December 7, 2004 10:08 PM | Permalink | Reply to this

NWS Responds!

The NDFDgenByDay method was just put up last week.

Count me as the first (soon to be happy) user, then.

Thanks for helping us to debug it! :-)

This was, as I indicated, a learning exercise for me, too. I really do like the idea of your NDFD-XML service. Good luck in your debugging efforts.

Posted by: Jacques Distler on December 7, 2004 10:41 PM | Permalink | PGP Sig | Reply to this

Re: Ill-formed NOAA

Bob, if you are open for suggestions, I would like to recommend that you place your XML directly into the output without escaping it again. This would allow languages like Java and .Net to generate classes which will directly consume your data.

Posted by: Sam Ruby on December 8, 2004 8:42 AM | Permalink | Reply to this

Re: Ill-formed NOAA

Thanks for the suggestion. We’ll look into this… are always open for suggestions and learning everyday.

Sorry for the delay in getting back.

Posted by: Bob Bunge on January 5, 2005 12:13 PM | Permalink | Reply to this

Re: Ill-formed NOAA

Sam (or Bob),

I am trying to use the NDFDgen service via Apache Axis. I generated the bindings and classes and have tried to invoke both methods. What I get is “Bad envelope tag: definitions”.

My best guess is that the tag is missing the “wsdl:” namespace. Is that possibly the problem? Other WSDLs that I am currently using have the wsdl: prefix on and many other tags.

Thanks for any help!

- Jack

Posted by: Jack Gould on January 5, 2005 4:00 PM | Permalink | Reply to this

Post a New Comment