1. MMS - a technology whose time has come

Not since the arrival of WAP has a wireless technology been so eagerly welcomed as having the potential to transform the way mobile phones are used. Although phones and wireless networks are only beginning to support it, the surge of interest in MMS from all parts of the wireless value chain - handset manufacturers, network operators, infrastructure providers, content providers, ISVs - is remarkable.

But why all this optimism? How much is based on reality and how much just on hype? Will MMS deliver to the expectations that have been raised, or fall short in practice like WAP before it? It is early days to answer these questions. But it is possible even now to enumerate some of the premises on which the success of MMS will depend, and to consider which are being fulfilled or are likely to be fulfilled. In particular we can ask whether MMS is susceptible to the kinds of problem with which WAP and, to some extent, SMS were beset.

1.1. More than just photo-messaging

First, it is important to distinguish between the user experience of MMS and the technology itself. It has been quite heavily marketed (at least in Europe) by handset manufacturers and network operators as photo-messaging, namely the ability to send a digital image directly from one handset to another and have it displayed at the other end. The image will often have been created by a camera integrated into the phone or else attached to it. And undoubtedly this usage will drive - indeed is already driving - substantial take-up of MMS-enabled handsets, taking MMS beyond the "early adopter" phase.

But this type of usage is only a lowest common denominator perspective on what MMS offers, colored by the default capabilities of the phones which currently support it. It ignores the much richer usage which becomes possible when the offerings of content and application providers are bought into the picture. For, MMS builds on the success not only of person-to-person SMS (adding as it does multimedia content); it takes advantage also of WAP push technology, allowing content to be delivered in a timely manner to the phone, rather than placing reliance on the navigation of tedious menu systems.

Some of the content might be in the nature of extensions to the person-to-person capability, adding customized audio tones, avatars, splash screens, etc. More innovation is likely to take place in terms of pushing multimedia content to the phone, e.g. a video clip of the latest goal by your favorite team, a new release from your favorite rock band, or stock market updates. But this is still perhaps only the beginning of what might be accomplished through MMS.

1.2. Beyond WAP and Web

The real potential of MMS to drive the demand for smartphones and 3G technology may not be directly from such server-driven services. Rather it may come from a combination of content and services pushed or pulled from a server with smart client programs which can display the multimedia content in a customized way and interact with the user. This is where MMS can potentially go beyond the scope of WAP and Web to create a new user experience.

With WAP and Web, diverse content can be sent, its (MIME) type recognized and the rendering and presentation carried out through a standard browser, with the possible assistance of plug-ins to handle specialized data types.

The data exchange is essentially driven by client pull. And the workability of the system depends upon the widespread availability of generic clients. The plug-ins, although nominally optional extras, need to be fairly standard to be usable, or at least easy to obtain and install.

On phones supporting MMS, there is likely to be a generic MMS-handling application or "MMS viewer" with fairly general capabilities to handle text, graphical, audio and (possibly) video content, whose presentation is controlled by a script in a language such as SMIL. The standard set of data types can be extended by custom plug-ins in much the same way as happened for Web browsers. But this approach is less than satisfactory for MMS. Few phones currently support more than a few basic data types. So matching the capabilities required by your advanced data service with those available on the target phone is likely to be a daunting task.

If your content can be handled by a standard MMS viewer with basic capabilities, that option is available. But suppose you want to send customised presentations out to smartphones which have advanced capabilities, or to allow some client processing of the data to occur and possible further requests to be sent out. Then the generic MMS viewer model breaks down and a custom solution is clearly needed. A potential solution to this problem of matching the requirements of a data service to the capabilities of a smartphone is to provide a custom client to ensure your content is handled optimally.

In other words you as programmer can, as circumstances require, resort to a client-server model. Rather than relying upon a "standard" MMS-handling capability embodied by the MMS viewer on the phone, you provide a client application which intercepts the incoming MMS message and provides its own UI built atop the MMS engine capability underlying the MMS viewer application. We will see below how Symbian OS already provides you with just such capabilities.

One of the clear advantages of this approach is it solves immediately the problem of how to provide a means for users to subscribe to your service, or to make a selection from amongst the customised services you provide; or better, to make successive requests. (At present subscriptions are often set up through a voice-based menu system, an approach which has obvious limitations.) It solves at the same time the problem of ensuring that clients only ask for content which their phone has the ability to handle.

Not only that, your smart client can also take advantage of other APIs available on the phone, like accessing a contacts database, creating a file locally to cache user preferences, or interrogating your server on start-up about new services of which the user could be notified, etc. Another convenient feature your custom application could add would be to force all MMS content requested during the current session to be downloaded to the phone without user intervention (see 2.1 and 3.3 below), thus providing a more seamless interactive messaging experience.

Indeed the whole user experience in this scenario would be of seamless integration of your service with their use of the phone as a whole, rather than their having any consciousness of "using" MMS technology.

As an example of what I mean, imagine the creation of a smartphone-based quiz game, where the user has to answer correctly a series of progressively difficult multiple-choice questions to win a prize. Rather than being textbased, the questions could be built around arbitrary multi-media objects:*  identify a famous person from a visual image  choose which of four images represents a named geographical feature  identify a singer or pop group from a brief audio clip say what happens next in a video identify the producer of a film clip The client application would use MMS parsing and rendering APIs to display the content (a limited number of formats for which would be planned in advance) and provide buttons to allow an answer to be selected. The answer sent back would be examined on the server. If it were correct, the next question would be sent, otherwise the game would be restarted.

1.3. The user as content provider

Although the main usage of MMS-enabled phones is likely to be, at least in the initial stages, for fairly simple tasks such as sending a graphic (or photo) with an accompanying message or caption, or else a video or audio clip, the usage is by no means so restricted. There is nothing to stop an end user creating their own fairly sophisticated script incorporating multiple media objects into a synchronised presentation. Indeed the level of sophistication achieved is limited only by their ingenuity and what is supported by the editor provided on the phone.

The possibilities increase if a suitable client application is installed on the phone capable of collecting subscriptions or requests. The phone user could then set him or herself up as a small-scale MMS content provider without subscribing directly to the services of a content delivery server. They could either send out multimedia presentations as these were created to a pre-defined list of subscribers, or else they might send out a pre-defined MMS presentation to clients making requests, say, by SMS.

Speculation about the kind of content that could be sent out in this way is left as an exercise for the reader. For example, a house seller or renter could create an MMS presentation showing off their property and post an advertisement in the local paper giving instructions on how to request for this presentation to be sent. Else, an estate agent could push presentations of new properties coming on the market to any potential buyers on their books.

1.4. Pitfalls MMS needs to avoid

Of course, despite the initial optimism, there are no guarantees that all is going to be plain sailing for MMS. Many of the hurdles which had to be crossed by SMS, Web and WAP have still to be negotiated by MMS. However there is the benefit of hindsight and the greater spirit of co-operation across the wireless industry which has undoubtedly arisen of late, in part through the learning experience with these technologies.

One of the first problems is network support and interoperability between networks. This delayed the takeoff of SMS quite badly, in particular because SMS is a GSM technology so not widely available in the Americas and not available at all in Japan. Nevertheless, equivalent technologies have filled this role (pagers in the USA and other chat services in Japan). And as it became clear to European operators that much more revenue would be generated by all by facilitating inter-network exchange of SMS, than by operating in exclusive mode, interoperability was fairly quickly introduced.

Although we are not seeing interoperability between competing networks in relation to MMS at present (in the few environments when more than one operator is offering the service), this is probably best explained in terms of logistical constraints as the technology is rolled out rather than as any lack of commitment to interoperability on the part of networks.

An initial problem with WAP was the early lack of availability of phones supporting it. (Indeed this led to the reinterpretation in the USA of the acronym WAP as meaning "Where Are the Phones?".) The initial poor take-up of WAP-enabled phones was undoubtedly due to the perception that there were few compelling WAP services around. And that situation was not improved by the lack of WAP phones. MMS looks to be in a position to avoid this kind of chicken-and-egg problem. Since camera-enabled MMS phones are attractive in themselves without any MMS-based services available, the time-scale for MMS phones to penetrate the market should depend only weakly on the timing of the rollout of MMS-based services.

A different problem beset WAP as the number of phones supporting it started to increase, which was that the number of different models on the market increased correspondingly. And all tended to have slightly different screen sizes, performance characteristics and capabilities. This was compounded by the announcement of new WAP specifications by the WAP Forum. While the presentation of MMS messages is likely to be less sensitive to screen size than that of WAP pages, MMS messages will still suffer from problems with differing device capabilities. In recognition of this the MMS Interoperability Group which includes major handset manufacturers such as Nokia, Motorola, Siemens and Sony Ericsson has established a basic standard enshrined in the MMS Conformance

This specifies the minimum MMS presentation capabilities which these manufacturers' MMS-enabled handsets will support. Of course content providers will wish to provide content which doesn't conform to these (fairly stringent) guidelines with a consequent downgrading of the quality of the average user experience. A suggestion for a strategy for dealing with this potential problem by matching devices to content commensurate with their capabilities was discussed above (see 1.2).

Another problem WAP had was the perception of poor latency, namely that it took a long time to set up a connection with a WAP gateway and navigate to and download the information you wanted. MMS is more immune to that difficulty, as are SMS and e-mail, since none of them was intended to provide real-time performance. Also with the arrival of 2.5G and 3G network technologies (which will be the main bearer for MMS) we can expect the inherent network latencies to be progressively reduced.

2. Overview of how MMS works

It is assumed here that the reader has some familiarity with how MMS works. For completeness, this section provides an overview is of the most important details and concepts to provide a context for the ensuing discussion of Symbian's MMS Client API. For more details about MMS, the reader is referred to the further reading in 4 below and references cited therein.

From the WAP MMS Architecture Overview (1 below):

"The Multimedia Messaging Service (MMS), as its name implies, is intended to provide a rich set of content to subscribers in a messaging context. It supports both sending and receiving of such messages by
properly enabled client devices."

We will look first at how MMS messages are sent and received, then at the structure of the messages and how they are constituted.

2.1. MMS Content Delivery

MMS can be thought of on one level as SMS but with the capability of sending a combination of multimedia objects and text messages. These messages are forwarded to a MMS server (the multimedia equivalent of a SMS Centre) which stores them until such time as they can be delivered to the intended recipient. Notification of an impending MMS message is sent at the earliest opportunity to the recipient. At this point the scenario diverges from the SMS one.

What happens next depends on a number of factors, in particular the kind of client. Options include:?? MMS client?? SMS client?? e-mail client If the recipient is a mobile phone with MMS capability, the notification will have been sent over a WAP bearer making use of WAP PUSH technology. A request can then be made, either immediately or at some later time, for the body of the message (which may be quite lengthy) to be sent to the phone. Once downloaded the MMS message can be freely viewed at any time.

If the mobile phone does not have MMS capability, the way in which the MMS content is made available is currently unspecified and will depend on the strategy adopted by the MMS server. For example, for terminals supporting SMS but not MMS, Nokia's Multimedia Terminal Gateway stores the MMS content in its internal storage and sends an SMS message to the intended recipient notifying them of the URL of a Web page which they can visit to view the MMS message using a suitable browser. A similar approach could be used to communicate with pagers.

Alternatively it is possible to send an MMS message to an e-mail address, in which case it will be delivered directly to the recipient's mailbox, without any prior notification being necessary. From there it can be downloaded and viewed in the standard way through a suitable e-mail client, either on a mobile phone or on any other Internet terminal. The way the client handles the data depends on the capability of the e-mail client program. If it is has support for handling the MIME type "multipart/related", it should be capable of presenting the separate multimedia objects and text which make up the MMS message. Otherwise, the e-mail client may make them available as detachable files. If it also has MIME support for the markup language used, for example "application/smil" (see 2.2 below), it should be able to display the content in the manner intended by the sender, or something approximating to it.

2.2. MMS Presentation

The structure of the MMS message itself is not specified in the relevant WAP Forum standard (see 3 below). The basic idea is that the message typically consists of a set of instructions, or script, controlling the presentation, and a collection of multimedia objects referenced by the script. For simple MMS messages there is an option to omit the script, in which case the viewer on the target phone (or other client) will just render the embedded multimedia object(s) as it sees fit. If a script is included there are a number of possibilities for the markup language used (see 1 below).

One possibility is that WAP Markup Language can be used. A more popular choice however is W3C's Synchronised Multimedia Integration Language (SMIL), which provides extended capabilities handling such things as timing of multimedia objects and animation, as well as the layout of text and graphical objects on screen. For example, let's say you are putting together a presentation with two captioned graphics, an audio clip and a video.

Using SMIL you could instruct that the MMS should be presented with the two captioned graphics one above the other while the audio clip was played. Then the video could be shown in a loop until it had run, say, 5 times, or the user terminated it. For more details on the capabilities and syntax of SMIL.*3. What can I do with MMS on Symbian OS v7.0?*So what support does Symbian OS currently provide for working with MMS? We will look briefly in the remainder of this paper at Symbian's MMS architecture and the kinds of things you can do with MMS through public APIs in

v7.0. In parts 2 and 3 of this paper, we will look in more detail at each of the areas discussed below, exploring the APIs themselves and providing some code samples. We will also compare these APIs with the corresponding APIs provided with the Series 60 SDK for Symbian OS v6.1. While these latter APIs are specific to Nokia's Series 60 platform, they provide much of the functionality available as standard in v7.0 of Symbian OS, and through a framework and an API which are not dissimilar to Symbian's.

3.1. Symbian's Messaging Architecture

The first important thing to understand about Symbian's MMS implementation is that it is not provided as a standalone technology but is closely integrated into the Messaging Architecture.

As we will see this provides certain key advantages, both to the programmer and in terms of the end-user experience, but it does add an additional layer of complexity in understanding how everything fits together. For that reason we'll talk a little bit about Symbian's messaging architecture to provide some context for the later discussion of MMS. Readers familiar with this topic can skip to the next section (3.2).

Symbian's Messaging Architecture is a powerful and extensible multi-protocol messaging framework. It allows the creation of plug-in modules to support individual messaging protocols such as fax, e-mail (POP3, IMAP4 and SMTP) and, most recently, MMS. The set of components that make up such a plug-in module is called a Message Type Module (MTM). All interaction with lower-level communication protocols, such as TCP/IP, is performed by the MTMs.

So when a message whose protocol type is handled by an MTM arrives on a phone, it will automatically be handled by the messaging architecture. From the user's perspective this means it will typically appear in the inbox or other appropriate folder in the messaging application from where it can be read and, if appropriate, replied to or forwarded. The MTM typically provides a UI framework which allows this, as well as allowing the creation and editing of new messages of that protocol type. The beauty of this from an end user's perspective is that messaging is an integrated experience rather than a series of disparate applications.

But the real power of the messaging framework from the perspective of the developer is that it provides hooks for third party applications to express interest in particular messages (for example SMS messages sent from a certain number or to a certain port), recover them from the message store and process them on the fly. This means that your application can take advantage of the messaging framework without the messaging application being explicitly invoked.

An MTM implementation for a particular protocol typically has the following main components:?? User Interface MTM - This provides for such things as displaying incoming messages, creating and editing new
messages, manipulating folders and their contents, and notifying of message transfer progress.?? Client-side MTM - This acts as the interface between the internal representation of a message's data and the
User Interface MTM, and allows encoding of recipient addresses, setting of subject lines, and validation of content.?? UI Data MTM - This makes commonly used aspects of the MTM like folder icons and MTM-specific menu options accessible without instantiating a User Interface MTM.?? Server-side MTM - The server-side component of an MTM deals with the interaction between the internal data
representation of a message, and the transport components. If you are only using, rather than implementing, an MTM, you will not need to concern yourself with this.

Client-side support for MMS on Symbian OS is provided through three main components:?? Client-side MTM (3.2 and 3.3 below)?? MMS utilities (3.4 below)?? SMIL parser (3.5 below) Notice that there is no user interface MTM. It is up to the implementer of a messaging client on a particular phone or reference platform to provide such functionality, to decide its scope and to document any public API exposed.

3.2. Creating and sending MMS messages

Clients can create and send new MMS messages. The steps to do this can be summarised as:

• getting an MMS client-side MTM
• creating a new empty message
• adding media objects
• setting message headers (such as recipients)
• finally, a specified message or messages can be sent. New MMS messages can also be created by forwarding or replying to existing MMS messages.

3.3. Receiving MMS messages

MMS Messages are delivered from a remote server to the Symbian OS phone in the following way:

• The remote MMS server sends a new-message notification (a WAP Push message) to the phone. The notification can be regarded as an incomplete message that contains the message headers, but not the media objects.
• The Symbian OS WAP Push component recognises a MMS new-message notification, and activates the MMS Notification plug-in.
• The plug-in decodes the notification and stores it as a new entry in the message server index. Client applications listening for incoming messages are then notified
• If automatic fetch is on, the plug-in tells the MMS server-side MTM to retrieve the full message, after which it decodes the message, and updates the message index entry created in the previous stage. Message server client applications can then read, display, etc. the full message. If automatic fetch is not on, your application can, on receiving notification of a message in which it is interested, opt to retrieve the full message through a client-side MTM method, then process or display it.

3.4. MMS utilities

In order to facilitate the creation and processing of MMS messages, a number of ancillary classes are provided by Symbian OS. Each of these encapsulates a particular aspect of MMS messaging and provides a series of associated utility methods. For example, a CMmsHeaders class encapsulates the headers associated with a MMS message. It allows setting and modification of such things as the message priority, recipients, content type, expiry date and message size. The headers for a given MMS message are recovered from the associated CMmsMessageobject with the Headers() method. This allows that the methods relating specifically to MMS headers can be isolated from the object representing the MMS message, so preventing the API of the latter from becoming too cluttered. Other utility classes include?? MmsAddressParser?? CMMsRecipient?? TMmsMessageMIMECategory?? CMmsMediaObjectall with obvious functions.

3.5. SMIL handling facilities

Finally, Symbian OS provides some utility classes for creating and parsing SMIL documents. These assist in the processes, respectively, of creating and displaying multi-component MMS messages. The three main classes represent?? a SMIL document?? a SMIL parser, and?? a SMIL composer. Each of these is derived from generic classes handling XML, of which SMIL is a special case.

The SMIL parser is used on a given SMIL XML file to turn it into a SMIL document. The latter is a Document Object Model (DOM), namely a series of nested elements, each represented by an appropriate class allowing its attributes to be accessed, modified and or removed. These elements encapsulate media objects and the layout and timing to be used in their presentation.

The SMIL composer on the other hand is used for the opposite task of creating a SMIL XML file to be sent in an MMs message. It is basically just a wrapper over the XML composer class from which it inherits and which provides the core methods for inserting tags representing the beginning and end of elements into an XML document and adding attributes to those elements.

4. Further reading

1. Wireless Application Protocol Multimedia Messaging Service Architecture Overview Specification (WAP Forum)http://www1.wapforum.org/tech/documents/WAP-205-MMSArchOverview-20010425-a.pdf

2. How to Create MMS Messages (Forum Nokia) http://forum.nokia.com/ndsCookieBuilder?fileParamID=2411

3. MMS Encapsulation protocol (WAP Forum) http://www1.wapforum.org/tech/terms.asp?doc=WAP-209\\\-
MMSEncapsulation-20020105-a.pdf

4. Synchronized Multimedia - SMIL 2.0 (World Wide Web Consortium) http://www.w3.org/AudioVideo/

5. MMS conformance Document (MMS Interoperability Group) http://forum.nokia.com/ndsCookieBuilder?fileParamID=2209 Symbian is headquartered in London, with offices worldwide. For more information see the Symbian website, http://www.symbian.com/. Trademarks and copyright 'Symbian', 'Symbian OS' and other associated Symbian marks are all trademarks of Symbian Ltd. Symbian acknowledges the trademark rights of all third parties referred to in this material. © Copyright Symbian Ltd 2002. All rights reserved. No part of this material may be reproduced without the express written permission of Symbian Ltd.