Accessing RESTful 'Astoria' Data Services in .NET Compact Framework

In my recent presentation at Microsoft Tech Ed EMEA 2008 in Barcelona (session MBL304), I talked about how to access RESTful web services from the .NET Compact Framework. I promised that I would post the sample code on my blog, so this post is to fulfil that promise, and also to explain how to do this for those who weren't in Barcelona. I also used this sample as an example of how to use HttpWebRequest in my recent webcast on Networking, so this post is also relevant to that webcast.

What is a RESTful Data Service?

REST stands for 'REpresentational State Transfer' and is a technique for exposing data over the web in a lightweight manner that is suitable for accessing programmatically using simple HTTP GET, PUT, POST and DELETE verbs. The actual resource that you wish to get/delete/update/create is identified by the uri of the request. For example, if you have a data service that exposes the data of the Northwind database, to get a list of the top-level collections in the data service (maps to the tables or views in the Northwind database), you might issue an HTTP GET to http://myserver/myNorthwindDataService/Northwind.svc and you'll get back a bunch of XML. To get a list of all the Customers in Northwind, you issue a GET to http://myserver/myNorthwindDataService/Northwind.svc/Customers and if you want a specific customer record, you can put the primary key in the URI like this: http://myserver/myNorthwindDataService/Northwind.svc/Customers('ALFKI') which returns all the data for that customer record as XML:

Creating an ADO.NET Data Services website

To create a site that exposes the Northwind database in this way, you need Visual Studio 2008 SP1 and the Northwind database installed in SQL Server 2005/2008 - I've included a script to create the Northwind database in the download accompanying this post. Follow these steps:

1. Create a new ASP.NET Website project called DataServicesWebsite - this should be hosted in IIS not the file System to allow devices to access it.
2. Delete Default.aspx as we'll be generating our own service
3. In Solution Explorer, right-click on the website root and then click Add New Item
4. In the Templates window, select ADO.NET Entity Data Model, and in the Name field, enter NorthwindModel.edmx. Click Add. Click Yes to the 'You are attempting to add a special file type...' pop-up.
5. In the Entity Data Model window, select Generate from database and click Next.
6. Click New Connection to make a connection to the SQL Server where you have Northwind installed, and when you have that setup click Next.
7. In the Choose Your Database Objects window, click the checkbox to select all the Tables, then click Finish.
8. You should now see the entity data model diagram - this model will provide the source for our data services website.
9. Right-click on the website root in Solution Explorer and then click Add New Item. Select ADO.NET Data Service on the Templates window, and then click Add.
10. You will now see a code editor window with the source of a class called WebDataService, which has a couple of TODOs in it. First you'll see that WebDataService inherits from a generic class: DataService< /* TODO: put your data source class name here */> - replace that TODO with the name of the data source class, which is NorthwindModel.NorthwindEntities. Then inside the InitializeService method, you need to set some access rules, which you can do easily by uncommenting the suggested lines and replacing the first parameter of each with a wildcard character. After this, the class should look like this:
    
    public class WebDataService : DataService< NorthwindModel.NorthwindEntities >
    {
        // This method is called only once to initialize service-wide policies.
        public static void InitializeService(IDataServiceConfiguration config)
        {
            config.SetEntitySetAccessRule("*", EntitySetRights.AllRead);
            config.SetServiceOperationAccessRule("*", ServiceOperationRights.All);
        }
    }
    
11. That's it! Now just right-click on WebDataService.svc in Solution Explorer, and click Set as Start Page. Now you can hit CTRL-F5 to run the site to check it works - you'll be able to fetch data from the database just by using a browser window. Try adding '/Customers' or '/Products' to the starting address in the browser window to see how it works.

 Creating a .NET Compact Framework Client

If you are lucky enough to be a desktop or Silverlight developer, then the ADO.NET Data Services team have created a client-side library for you to use called System.Data.Services.Client, but for some unfathomable reason, they didn't create this library for .NETCF. That library gives you what is sometimes known as 'LINQ to URI', as it translates LINQ queries into the required HTTP calls to fetch the data. So if you want to know how to use that library from a desktop or Silverlight client, you can look at this blog from Guy Burstein.

However, we mobile devs must do it the hard way - which actually is not so hard if we use some simple calls using HttpWebRequest and HttpWebResponse to fetch the XML, and a little bit of LINQ to XML to extract the data we want from the response.

First, we need to get the XML from the service. We can use a couple of utility methods to do this. Here's the entire class containing both methods:

using System;
using System.Text;
using System.Net;
using System.IO;

namespace RESTLibrary
{
     public enum HttpMethods {
      GET = 0,
      PUT,
      POST,
      DELETE,
      HEAD,
      OPTIONS,
      LIST,
      UNKNOWN }  
   
    public class REST
    {
        public static HttpWebRequest CreateRequest(string uri, HttpMethods method, string data, string contentType, string userName, string password)
        {
          WebRequest request = HttpWebRequest.Create(uri);
          request.Method = Enum.ToObject(typeof(HttpMethods), method).ToString(); 
          request.Credentials =
            new NetworkCredential(userName, password);
          request.ContentType = contentType;
          ((HttpWebRequest)request).Accept = contentType;

          if (method != HttpMethods.GET && method != HttpMethods.DELETE)
          {
              Encoding encoding = Encoding.UTF8;
              request.ContentLength = encoding.GetByteCount(data);
              request.ContentType = contentType;
              request.GetRequestStream().Write(
                encoding.GetBytes(data), 0, (int)request.ContentLength);
              request.GetRequestStream().Close();
          }
          else
          {
              // If we're doing a GET or DELETE don't bother with this
              request.ContentLength = 0;
          }

          // Finally, return the newly created request to the caller.
          return request as HttpWebRequest;
        }

        public static string ReadResponse(HttpWebResponse response)
        {
            // Read the contents to a string
            // and return that to the caller.
            string responseBody = String.Empty;
            using (Stream stm = response.GetResponseStream())
            {
                using (StreamReader reader = new StreamReader(stm))
                {
                    // Simply read in the entire response to our string.
                    responseBody = reader.ReadToEnd();
                    reader.Close();
                }
            }

            return responseBody;
        }
    }
}

The first of these methods, CreateRequest, creates an HttpWebRequest instance properly configured to make the call to fetch the data. The second utility method, ReadResponse, reads the string that is sent back from the web data service and returns it to the caller. The client code to use these looks like this:

            // Create the request object using the utility method
            // This code accesses host ppp_peer so the device must be connected to the server by ActiveSync.
            HttpWebRequest request = REST.CreateRequest(@"http://ppp_peer/DataServicesWebsite/NorthwindService.svc/Customers",
              HttpMethods.GET, String.Empty, @"application/atom+xml", "", "");

            // Now, attempt to read in the data.
            using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
            {
                string responseXml = REST.ReadResponse(response);
                
                // Do something with the response XML
                ...
            }

All that remains is to extract the data we want from the XML. If you haven't used LINQ to XML much, here is the important stuff you need to know.

Let's say we want to extract the CustomerID, CompanyName and City of every customer in the Customers collection. If you use a browser to look at the XML returned for the Customers collection, you'll see that the elements we want are all children of the properties tag:

<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
- <feed xml:base="http://localhost/DataServicesWebsite/NorthwindService.svc/" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="http://www.w3.org/2005/Atom">
  <title type="text">Customers</title>
  <id>http://localhost/DataServicesWebsite/NorthwindService.svc/Customers</id> 
  ...
- <content type="application/xml">
- <m:properties>
    <d:CustomerID>ALFKI</d:CustomerID>
    <d:CompanyName>Alfreds Futterkiste</d:CompanyName>
    <d:ContactName>Maria Anders</d:ContactName>
    <d:ContactTitle>Sales Representative</d:ContactTitle>
    <d:Address>Obere Str. 57</d:Address>
    <d:City>Berlin</d:City>
    <d:Region m:null="true" /> 
    ... 
  </m:properties>

The m: prefix to the properties tag, and the d: prefix to the individual fields tags are important - they indicate that the tag belongs in a particular XML namespace. The 'd' and 'm' are both symbols for namespaces, which are defined at the top of the XML in the <feed> element:

- <feed xml:base="http://localhost/DataServicesWebsite/NorthwindService.svc/" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="http://www.w3.org/2005/Atom">

Tags that do not have a prefix are in the default namespace, which is the one specified in the <feed> element that does not have prefix, which is the last one listed, xmlns=http://www.w3.org/2005/Atom.

Once you understand the definition of the namespaces, the LINQ to XML query to select the required customer properties from every record in the Customers collection becomes the following. This example not only selects the target fields, but uses the values to initialise a new instance of a CustLite object, which is a custom class defined for this example that just has three public properties, CustomerID, CompanyName and City. As a result, the type of the customerIdList, defined as var in the code, is actually a List<CustLite>:

                // After reading in the response parse the response into a XDocument.
                XDocument customerDoc = XDocument.Parse(responseXml);

                XNamespace ads = @"http://schemas.microsoft.com/ado/2007/08/dataservices";
                XNamespace m = @"http://schemas.microsoft.com/ado/2007/08/dataservices/metadata";

                var customerIdList =
                  from c in customerDoc.Descendants(m + "properties")
                  select
                    new CustLite()
                    {
                        CustomerID = c.Element(ads + "CustomerID").Value,
                        CompanyName = c.Element(ads + "CompanyName").Value,
                        City = c.Element(ads + "City").Value,
                    };

Once we have our List<CustLite> we can use it in our app, for example to display it in a DataGrid:

You can download the code for the sample from the link below. Note that this example shows how to get data formatted using the ATOM MPP XML format, but ADO.NET Data Services also returns data in JSON if you ask it right - that's the subject for another post...