WSO2 Mashup Server - What can it do for you?

If you were wondering what the WSO2 Mashup Server can do, here is a nice little flash demo that explains that very thing in a couple of minutes.

WSO2 Mashup Server 1.5.2 Released

Its been some time since we released the 1.5.1 version of the WSO2 Mashup Server. The main reason for doing the 1.5.2 release was a memory leak that was present in the release which was inherited from Axis2. This memory leak was fixed in the 1.41 release of Axis2 but we could not focus on doing a release because of the effort that was put into WSO2 Carbon.

So Mashup Server users should upgrade to the latest version. The upgrade is pretty straight forward and is just a matter of preserving the scripts and the database folder of the old installation.

In addition to this memory leak there were a few other issues that we fixed. You could have a look at the release note for full details.

WSO2 Mashup Server Resources

If you are looking for resources on the WSO2 Mashup Server, make sure you pay a visit to its library page on wso2.org. It summaries all the available resources.

Invoking secured services couldn't be easier

A key feature of the 1.5.1 version of the Mashup Server that we released mid last year is that it can invoke services secured using complex WS-Security scenarios with ease. Hear is an example of it,

The service securedService running up on mooshup.com has been secured such that the request sent to this service should be encrypted and all requests should contain the username and password in a SOAP header (requires Username Token). Hence all registered user on mooshup can access this service. A client for this scenario would be simple as,

invokeSecuredService.inputTypes={"firstParam" : "string" , "secondParam" : "string"};
invokeSecuredService.outputType="string";
function invokeSecuredService(firstParam, secondParam) {
  var request = new WSRequest();
  var options = new Array();
  options.encryptionUser = "keithspublickey";
  options.username = "yourUsername";
  options.password = "yourPassword";
  var payload = {firstParam}{secondParam};
  var result;
  try {
      var service = new QName("http://services.mashup.wso2.org/securedService", "securedService");
      request.openWSDL("http://mooshup.com/services/keith/securedService?wsdl", false, options, service, "SOAP12Endpoint");
      request.send("demo", payload);
      var response = request.responseE4X;
      result = response["return"].toString();
  } catch (e) {
      system.log(e.toString(),"error");
      return result = e.toString();
  }
  return result;
}

You could follow the "Invoking Web Services from a Mashup" tutorial to get a feel of how easy it is to access web services using the WSO2 Mashup Server.

Invoking Web Services from a Mashup

An article written by me titled "Invoking Web Services from a Mashup" has been published on the WSO2 Oxygen Tank. Invoking external web services is a popular mechanism to get information to mashup with and this articles explains how this could be done on the WSO2 Mashup Server. The article covers the following,

• Mechanisms that the WSO2 Mashup Server provides to invoke web services
• The pros and cons of each mechanism
• An example showing how each of these mechanisms could be used to invoke a web service in a synchronous as well as asynchronous manner. It also gives an example as to how a secure service could be accessed.

WSO2 Mashup Server ScreenCast #3 - Service Composition

The third of Jonathan's screen cast on the Mashup Server is now out. This edition focuses on Service Composition using the WSO2 Mashup Server. It shows how easy it is to compose services as well as demonstrates the use of E4X constructs when writing mashups.



The first edition was a helloworld followed by how the mashup server could be used to do web scraping. You could have a look at the high-res version on the WSO2 Oxygentank at,
ScreenCast - Hello World
ScreenCast - Scraping
ScreenCast - Service Composition

Speaking at IITC 2008 - Server-Side Mashups for enterprise SOA

I will be conducting a half-day tutorial titled "Server-Side Mashups for enterprise SOA" at the 9th International Information Technology Conference (IITC 2008) on the 30th at the University of Colombo School of Computing. This will be a beginner level tutorial where I will talk of how the WSO2 Mashup Server can be used as an enterprise integration tool.

WSO2 Mashup Server Screencast #2 - How to Scrape a Web Page

In the previous screencast Jonathan explained how you could get started with the Mashup Server with a simple Hello World Mashup. Now the second in this series of Screencast is out.



In this screencast Jonathan explains how you could scrape a web page and make that information available to tour mashup. There is a whole load of information available on the web that is not accessible via an API and screen scraping can be used to get that information into your mashups.

You could view the Hi-Resolution version of this if you have good bandwidth.

WSO2 Mashup Server - Quick Start Guide

Channa has composed a nice little Quick Start Guide for the Mashup Server. It shows how you could get started by installing it and provisioning it and the later section cover how you could get data from a external web page using the scraper tool, how you could access a external web service using a generated stub and how you could read in a feed using the FeedReader.

Its a nice little pack that gets you started off pretty quick.

WSO2 Mashup Server Screencast - Getting started with a HelloWorld

The WSO2 Mashup Server is a platform for hosting mashups written in JavaScript with the E4X extension. These mashups can consume data from various sources such as feeds, HTML pages, external Web services, databases (via Data Services), Excel files (via Data Services) etc. These newly created mashups are then exposed as Web services in return.



In this Screencast, Jonathan Marsh, Director of Mashup Technologies for WSO2, demonstrates how to create a simple hello-world Web service using only the WSO2 Mashup Server and Notepad. A simple Javascript function is deployed as a Web service, WSDL and other metadata is automatically generated, and then the service is tested using the Mashup Servers try-it page. The service also shows how to quickly construct XML using the E4X Javascript extensions, and how to apply simple annotations documenting the operations and declaring parameter types.

HTTP Content Negotiation with Apache Axis2

Since the 1.3 release of Axis2, it supports HTTP Content Negotiation. It is not turned on by default but can be done so by adding the following parameter to your axis2.xml.

<parameter name="httpContentNegotiation">true</parameter>

Axis2 does not honor the content-type text/xml when its in the Accept Header. This is because the content-type text/xml is primarily used for SOAP 1.1 but when a request is sent via the browser it has the Accept header as "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 ". If we were to honor text/xml this would result in a SOAP 1.1 response. Not too good for a REST scenario ha! This is the result of the same content-type (text/xml) been used for two message formats (SOAP 1.1 and POX).

Now lets get back to HTTP Content Negotiation with Apache Axis2. Performing a GET request on the version service would result in the following request and response. You could capture them by using TCPMonitor.


Now try editing the Accept header to "application/json/badgerfish". You should have enabled JSON support for this to work. This would result in,


A blog post on how to enable JSON on Axis2 will follow. This configuration can be used to enable HTTP Content Negotiation on WSO2 WSAS as well as the Mashup Server. It should work with Synapse and the WSO2 ESB too but I haven't checked it.

Mashup Server as a Teaching Tool

I've been involved in quite a few trainings in the recent past. Both on Axis2 as well as the Mashup Server. The newest on the list was a "Introduction to WSDL 2.0" training that I conducted yesterday. One of the cool tools that I have used in all these trainings is the Mashup Server. The Mashup Server makes it amazingly simple to to create services in a matter of minutes, whats more there is a bunch of resources thats auto-generate for each service.

For e.g I used my RESTDemo mashup extensively to describe how WSDL 2.0 enables REST.

Note : The RESTDemo Mashup does not cover all capabilities of WSDL 2.0. I intend to do a comprehensive write up of this soon.

One might wonder how the Mashup Server helps teach a person Axis2? Well to be frank the Mashup Server is a highly customized version of Axis2. We've used quite a few extension points in Axis2 to make the Mashup Server what it is. Hence its a great case study for the customizability of Axis2.

Expose and Invoke Mashups over JMS

My article on Exposing and Invoking mashups using SOAP/POX over JMS with the WSO2 Mashup Server has been published on the WSO2 developers portal for SOA. It talks about how you could invoke mashups running on a WSO2 Mashup server over JMS and also how you could send JMS messages using a Mashup running on the server. You would me amazed to see how easy it is. The article contains sample code which help you get going in minutes.

JMS plays an important part in most enterprises and this open the door for enterprise mashups using the Mashup Server.

Get Twitter Friends Time Line Using a Mashup

Milinda has posted how to Get Your Twitter Friends Time Line Using libcurl. I thought of publishing a alternative mechanism to doing it. How about getting your twitter friends time line using the WSO2 Mashup Server? Well is pretty simple and straight forward. Here is the code (Just a few lines of code),

function getTwitterFriendsTimeline(){
   var request = new WSRequest();
   request.open("GET","http://twitter.com/statuses/friends_timeline.xml?count=5", false, "username","password");
   request.send(null);
   return request.responseE4X;
}

Pretty simple ha! If you wanted you could have published this on mooshup and have it run for you 24*7 for you. Of course you might want to do something with this data to make it a useful mashup.

REST and WSDL 2.0 Discussion continues

In response to my blog post on using WSDL 2.0 to describe RESTfull services and Sanjiva's challenge Stefan has put forward 5 questions for me. Here are the answers to them.

Question: How does one one describe representation formats other than XML?
Answer: This can be done using the whttp:inputSerialization (for input messages), whttp:outputSerialization (for outgoing messages) and whttp:faultSerialization (for outgoing faults). The WSDL 2.0 Adjuncts spec defines three types of serializations. They are application/x-www-form-urlencoded, application/xml and multipart/form-data. Axis2 supports all three of these as well as JSON messages (This is how you can send JSON messages using the Mashup Server). At WSO2 we are considering doing a JSON binding for WSDL. If we had that in place even this could be described in a standard manner.

Question: How is content negotiation supported?
Answer: WSDL cannot describe this (neither do I expect it to), but we do have content negotiation support in Axis2. Due to some practical limitations we had to make it an optional feature. This can be made available by adding the following parameter to the axis2.xml.

<parameter name="httpContentNegotiation">true</parameter>


Question: How does one describe whether a URI parameter is a query parameter, path element, or matrix parameter?
Answer: There are a few properties and defaulting rules that help you achieve this. As my example shows parameters can me made to go in the path or query section using templates. An example template for sending the parameter "name" in the path segment would be "student/{name}" while sending a parameter as a query parameter would be "student?name={name}". This section in the adjuncts spec give a brief idea as to where the parameters not cited would appear. If application/x-www-form-urlencoded serialization is used on a GET all parameters not cited in the template would go in as query parameters. There is a additional property in WSDL 2.0 that helps you control this too. If you specify whttp:ignoreUncited="true" then all uncited parameters will be dropped.

What do you really mean by matrix parameters?

Question: Is there a way to describe response codes?
Answer: WSDL 2.0 does not allow you to specify the http response code of a outgoing message. Too bad we missed this (I wonder whether the working group had discussed this issue though). I guess this is the one thing that WSDL 2.0 missed. Hence I opened a issue against the spec. But WSDL 2.0 allows you to set http codes on fault messages. For e.g on a fault you could say whttp:code="402".

Question: What about describing hypermedia? :-)
Answer: In my example you would have noticed that I actually return a set of links to where the actual resource exist when invoking some operations. For e.g getAllCustomers would return links to the actual customers. I agree that there is something missing in here, a way to describe that accessing this link returned, would fetch you the customers details (Probably an annotation in the schema could describe this).

Stefan had mentioned the following in his post

"One of the interesting aspects is the try out page, which is generated from the WSDL. Essentially this means that metadata is used to generate the self-descriptive form; somewhat backwards from a REST perspective, but understandable given that the goal was to have an external description in the first place."

Yes the WSDL captures all the metadata of your service. Don't you think that there is a definite advantage in having a standard description for your service? This would enable users to build tools that would make life easy when accessing such services. The tryit is an good example of this. You could even build a UI for a service using a generated JavaScript stub. The UpgradeChecker sample on mooshup is a good example for this.

Its good to have a constructive discussion around REST, SOAP and WSDL rather than try to proclaim that one of them is the way to go and the other does not have a place.

Sending IM Messages via Yahoo using the Mashup Server

[Note: also published on the Mooshup.com blog.]

If you try to send IM messages using a yahoo account in the WSO2 Mashup Server you may encounter the following error.

Fault: Cannot find yahoo library in the class path. Please make sure its in the classpath.

The reason for this could either of the following,

• The yahoo Library in not in your classpath - Download the latest version from YMSG
• You have the yahoo library in your classpath but its an old version. You need to have the latest version in your classpath to get this working.
  

Now that you have the latest version of the Yahoo messaging library in your classpath its time to test it out.
Here is a simple client for sending a IM message via yahoo,

function IMExample(){
   var im = new IM("yahoo");
   im.login("username","password");
   im.sendMessage("keithgodwinchapman","Hi, This was sent from the WSO2 Mashup Server!");
   im.disconnect();
}

For more details on sending IM messages using the WSO2 Mashup Server please refer the "Sending Instant Messages via Mashup Server" article.

You may have noticed that this sample is very similar to Sending MSN messages using the Mashup Server.
It is indeed the only difference is the argument you pass into the constructor of the IM host object. Have fun sending IM's using the WSO2 Mashup Server!

RESTfull Mashup with WSDL 2.0 - WSO2 Mashup Server

I was looking around for REST stuff related to Axis2 and stumbled upon one of Stefan Tilkov’s blog post (An old one at that). In that post Stefan had asked somebody to show a WSDL 2.0 document that describes this RESTfull scenario.

Too bad I didn't come across this back then, but as the saying goes better late than never. WSDL 2.0 can describe this scenario easily and I thought of writing a WSDL 2.0 document for this scenario (I've made one change to the URLS used in the scenario, instead of customer/{id}/orders I've used customerorders/{id}. This is not due to a limitation in WSDL 2.0 but rather the implementation of Axis2).

I put all these details together and built a little Mashup (which I named RESTDemo) that runs on the WSO2 Mashup Server. Of course the WSO2 Mashup Server has the ability to build RESTfull Mashups. In order to make it accessible to others I've uploaded it to Mooshup.com our community site.

Note : I have made some assumptions on how an Order and a Customer will be described. My assumptions is that an Order is described by an id (String), customerID (String), details (String), amount (Double), items (String array). A customer will be described as id (String), name (String), address (String), email (String).

The main objective of putting this Mashup together was to demonstrate that this RESTfull scenario can be described using WSDL 2.0. The WSDL 2.0 for this mashup can be accessed from the Mashup page, you could choose either the annotated view or the raw view.

The all important section in the WSDL is the HTTPBinding , cause this is where all the information for accessing this service via REST lies. Ideally I would have loved to make the content-type (whttp:inputSerialization in WSDL 2.0 terms) of a few of the operations application/x-www-form-urlencoded (WSDL 2.0 defaults it to application/xml) . But unfortunately as of now the Mashup Server does not offer a mechanism to do this. I've already opened a JIRA so that this could be added.

You might also want to try out our RESTfull mashup. Simply goto its tryit page in order to try it out (The tryit page allows you to test your mashup and a drop down at the top allows users to select the endpoint they want to access, in our case we may prefer to choose the HTTPEndpoint cause this is all about REST). Now this is where the benefit of a standard description (WSDL 2.0) for our RESTfull mashup kicks in. The tryit page is auto generated off the WSDL 2.0 doc (It has all the information as to how this mashup can be accessed, let it be SOAP 1.1, SOAP 1.2 or REST). The tryit page invokes the service using a auto generated JavaScript Stub.

Let me now explain as to why I had to change the URL mapping for get all customer orders. Axis2 (Which is what the WSO2 Mashup Server is built on) dispatches to the correct operation (when REST is used) using the http method and the URL of the request (The service is always found using the request URL). Now when we try to figure out the operation the request is headed to we basically do some pattern matching, and thats done against the first constant part we get off the httpLocation. In the scenario given it allows a GET on customer/{id}/orders as well as customer/{id}. Now if I used these two templates as it is, there would have been dispatching issues, hence I decided to change the second template.

This has become quite a long blog post. If you are interested in knowing how WSDL 2.0 can be used to describe RESTfull services stay tuned to my blog. I plan to write more on this topic in the time to come. Also there is a RESTfull services on Axis2 tutorial which is just around the corner.

Note : Mooshup is running the 1.5.1 version of the WSO2 Mashup Server (released before Axis2 1.4.1). There fore its REST URLs are of the format http://mooshup.com/services/keith/RESTDemo.HTTPEndpoint/customer/123, in the next version which will be based on Axis2 1.4.1 the URLs of the form http://mooshup.com/services/keith/RESTDemo/customer/123 would also work.

Two Years at WSO2

The beginning of this month marked the completion of my second year at WSO2. Its been a great two years at that. Initially I was involved in helping improve the Axis2 IDEA plugging for the 1.1 release. This was my first steps with Axis2 and that helped me gain the committership for Axis2 later that year. The Axis2 1.1 release was a very important one and one of my batch mates Thilina was the release manager.

My involvement with Axis2 was propelled by the fact that I got the opportunity to attend the WSDL 2.0 interop in France which was held from the 14th to the 18th of November that year.

Members at the WSDL 2.0 interop.

This was just 2 months after I had joined WSO2. I went to the interop along with Chinthaka. We had the chance to pay a visit to the Eiffel tower as well as the world famous Louvre museum.
.

At Eiffel

At Louvre

By the time we went to the interop I was not too familiar with the internals of Axis2, I was just providing a helping hand to Chinthaka. But that trip threw me into the deep end of Axis2. The time frame given to the working group was to end on the 31st of December and they needed two independent implementations to pull the spec through. We were one of the implementations and we had more work to do, especially on the http binding. The other implementation was from canon and they were not interested in implementing the http binding (They did support us by implementing a subset of the http binding though) so we had the added pressure and to make matters worse (or you could look at it as provide me more opportunity ) Chinthaka was leaving for grad school early December. So I was charged with the responsibility of implementing these stuff. That was how I got into the internals of Axis2.

Although I worked on Axis2 full time at the start later on I moved on to work on the WSO2 Mashup Server. Even though I've been working on the Mashup Server ever since, I've always been in close proximity with Axis2. That has been for 2 main reasons. One is that I love working on Axis2 and the second been the fact that Axis2 is the foundation of most of the products at WSO2 including the Mashup Server.

Invoking Mashups using JSON

Mashups running on the WSO2 Mashup Server can be invoked in using various message formats. They are SOAP 1.1, SOAP 1.2, POX (Plain Old XML), REST, JSON, SOAP/POX over JMS, SOAP/POX over SMTP.

Here is how you can use JSON to invoke a simple Mashup. In this example I will be invoking the RESTSample running on mooshup.com. This simple sample sends the payload as JSON and receives the response as JSON as well.

// Providing optional annotations
invokePOSTWeather.inputTypes={"city" : "string", "weatherDetails" : "string"};
invokePOSTWeather.outputType="#raw";
function invokePOSTWeather(city, weatherDetails){
     var request = new WSRequest();
     var options = new Array();
     // Instracts the server that my request needs to be formatted as JSON
     options.HTTPInputSerialization = "application/json/badgerfish";
     var payload = <POSTWeather><city>{city}</city><weatherDetails>{weatherDetails}
</weatherDetails></POSTWeather>;
     var result;
     try {
             request.open(options,"http://mooshup.com/services/samples/RESTSample/", false);
             request.send(payload);
             result = request.responseE4X;
     } catch (e) {
             result =  e.toString();
     }
     return result
}

If you were to use TCPMonitor to trace the messages you will be able to see the messages sent and received by the Server.

Here is the request sent out by this Mashup

{"POSTWeather":{"city":{"$":"col"},"weatherDetails":{"$":"10"}}}

And the response it received

{"ws:POSTWeatherResponse":{"@xmlns":{"ws":"http:\/\/services.mashup.wso2.org\/RESTSample?xsd"},"return":{"$":"col"}}}

Note : There are three main ways that you can get the response from WSRequest. There are response.responseText, response.responseXML and response.responseE4X. If you were to use these variants when the response if JSON they will act as follows,

response.responseText - Would return the JSON string as the response
response.responseXML - Would return the XML representation of the JSON string
response.responseE4X - Would return the E4X representation of the JSON string

Sending IM messages via MSN using the WSO2 Mashup Server

[Note: also published on the Mooshup.com blog.]

If you try to send IM messages using a windows live account you may encounter the error reported in the JIRA MASHUP-1087. The reason for this is that the library we use to send IM messages via MSN (jml) is outdated. A newer version of this library was released on the 5th of August 2008 and is available here.

In order to get this working you need to download the 1.0b3 version of jml and drop it into the lib/patches folder of the WSO2 Mashup Server and restart it.

Here is a simple client for sending a IM message via MSN,

function IMExample(){
var im = new IM("msn");
im.login("username@live.com","password");
im.sendMessage("keithgchapman@hotmail.com","Hi, This was sent from the WSO2 Mashup Server!");
im.disconnect();
}

For more details on sending IM messages using the WSO2 Mashup Server please refer the "Sending Instant Messages via Mashup Server" article.