Transforming the Web


With my brand new Nokia 6630 burning a hole in my pocket, I decided to build some infrastructure to help out mobile browsers.

There are a bunch of problems with browsing the web on your mobile phone right now and I decided I would reduce the impact of some of these issues. First, the screens are small, so content needs to be formatted for the screen. Second you can't type well on the phone keypads, so if I have to type in a URL I want it to be easy to type. Finally, much of the information I want to get on my mobile phone should be digested and extracted from web at a moments notice so I don't have to navigate or download full pages to get a small piece of information. What I decided to do was to build a server that would let you define a source of information (either HTML or XML) and then process that information using an XQuery transformation and then give you a short little URL that is easy to put on a phone. Here is the XQuery web transformation service. You'll notice that the hostname is mjg.md. This is only 6 keystrokes on a typical phone keypad, i.e. 654763. When creating a new service it will also generate a similar shortname for the service and you can either choose to use that or not, much like tinyurl for the web. Perhaps I should also offer that simple service as well.

Let's look at one of the examples that I wanted to use myself. When I drive to and from Palo Alto to Accel Partners, I like to get an idea of what I am in for on the road and I may not have time to check the computer, and it is dangerous for me to try and surf the web in my car on my phone. However, if I could quickly and succinctly get a listing of all the incidents on Bay Area highways without download a huge page and scrolling around, I would be pretty happy. Here is the page that has the information that I want, Bay Area Traffic. If you go to that page, you'll see it has a lot of header and navigations components and about 16 images. All of which I don't really need to get the information that I want. All I really need is that little table in the middle with the highways and descriptions of incidents. Using mjg.md I can create a new query, put in a title and a shortname, and process that web site so it doesn't have so much extra info. First I put in the URL, then I have to write some XQuery to pull out the relevant pieces. I typically do this via trial and error using the "test" function. Here is my first cleaned up version of the information:

<table>
{
for $row in //table[3]//table[3]/tr
  return <tr>{$row/td[2]} {$row/td[3]} {$row/td[4]} {$row/td[5]}</tr>
}
</table>

That makes for a simple page, though it isn't as nicely formatted for the vertical screen form factor that most phones have. We can go even deeper by moving around some of the elements so that the first 3 pieces of information are aligned vertically:

<table>
{
for $row in //table[3]//table[3]/tr
  return <tr>
{<td>
{<table>
{<tr>{$row/td[2]}</tr>}
{<tr>{$row/td[3]}</tr>}
{<tr>{$row/td[4]}</tr>}
</table>}
</td>}
{$row/td[5]}
</tr>
}
</table>

This gives us a page nicely formatted for the aspect ratio typically found on smart phones. Here is a photo of what that looks like on my Nokia 6630:

traffic
Not the greatest photo, but I found it pretty hard to take good shots of the screen. Maybe someone can point me to a good screenshot program for mobile phones. The page is also less than half the size of the original page and thats not even including the images (with images, it is less than 1/3 the size of the original). Feel free to experiment with the service as much as you want, but please don't do anything that makes me want to take it down or spend a week adding security to something that took a few hours to create.

For developers:

The service is running on Tomcat 5.0 and is completely written using JSP 2.0 and the JSTL. There are no scriptlets in the JSP pages and all code is abstracted into .tag files. It makes use of the Saxon 8 XQuery/XSLT engine for transformations, the JTidy library for cleaning up HTML pages, the DisplayTag library for showing tables, and PostgreSQL for storing all the data. The whole thing was built using IntelliJ 4.5.3 with my JSP 2.0 enhancements. If you have any questions or want to add some features to it, I can give you CVS access with good google references.

Posted: Thu - January 6, 2005 at 03:58 PM           |

©