Monday, June 14, 2010

Building an Enterprise App Store with WSO2 Gadget Server


Written by Tyrell

In my previous post, Portals and Mashups in the Cloud, I described an ecosystem that can be deployed either in your data centre or in the cloud.

An ecosystem such as above will allow enterprises to do two important things;
  1. Expose APIs for third party mashup/gadget developers to utilise in their apps
  2. Maintain a repository where business users can visit, browse and select apps from. These apps run in their personal portal pages.
Within that post lies the fundamentals of an Enterprise App Store. What is an enterprise app store? Well, in the mobile device world of iPhones, Androids and Symbians, we all use an app store at one point or the other. Whether it's iTunes, The Android Market Place or Nokia's OVI Store, we all visit them and they provide one significant benefit in general. We get to personalise our device with the applications we need. We get to build, at a software level, a phone that is optimised for our day-to-day use, without being programmers. We also get to influence the programmers who write these apps, by requesting new features for existing apps or, at times, encouraging them to develop new ones. The role of the device vendor here is to provide the best API possible for the app programmers. The vendor benefits by increased user base and the programmers get intrinsic and more often financial rewards for their creations.

An Enterprise App Store brings this to your organisation's IT department.

"The premise of an app store model for enterprises is simple: By removing the middleman, the famous bottleneck between the business and IT demand can be reduced in many cases. Application backlogs can shrink, consumption of internal and external IT resources will increase, and fierce competition to provide the best solutions to niches can greatly improve overall quality (the long tail of IT argument), all while reducing costs. At least, that’s what is possible if we look at what’s happening to the non-enterprise software market today." - Source, Dion Hinchcliffe.
This is why we had an Enterprise Gadget Repository in the WSO2 Gadget Server from day one. If you already run it within your enterprise (either in your data centre or in the cloud), you already have the infrastructure for an Enterprise App Store.

Adding gadgets to the repository.
An administrator can add gadgets to the repository in a few easy steps (we have documented how).

How users interact with the App Store.
Users sign in to the portal and see a default set of gadgets configured by the administrator. Of course they can change the layout of these gadgets and save their personal preferences for each. They can add new tabs, clone existing tabs to build new tabs.. but I digress. Let's see how they interact with App Store. When they decide to add gadgets, they are re-directed to the gadget repository.

Apart from browsing the directory and adding gadgets to their portal pages, the users get to rate and comment on them.

This builds a community where users see what others think about the app, feature requests and even bug reports. They also see how many are already running this app.
"That is, the app store supports an ecosystem of developers and creators, but acts as a governance mechanism to make sure the crappy and malicious stuff doesn't degrade and contaminate the ecosystem." - Source, Joe McKendrick.
A nice software delivery mechanism for SOA isn't it? :)

Additional Resources:

See original post

Sunday, June 13, 2010

Carbon 3.0 P2 Repository

Carbon 3.0 P2 Repository is located at

Saturday, June 12, 2010

New Configuration Model of Synapse


Written by Hiranya Jayathilaka

If you are trying out a latest Apache Synapse build off the SVN trunk, you will notice that the Synapse configuration model has gone through some significant changes lately. In the past, Synapse was programmed to load the entire mediation configuration (sequences, endpoints, proxy services, tasks etc) from a single XML file, named synapse.xml. Synapse would parse this XML file at startup and construct an object model known as the SynapseConfiguration which contains the definitions of all active mediation components at runtime. While this approach was simple and clean, it had a number of limitations:
  • Configuration management becomes a nightmare as the size of the synapse.xml grows. A typical heavy duty configuration would consist of dozens of proxy services, sequences and endpoints. When all these are packed into a single XML file, it becomes difficult to locate a single configuration item quickly.
  • With all configuration artifacts stored in a single flat file, it is almost impossible to develop satisfactory tooling support for configuration development and maintenance.
  • A team of developers cannot work on configuring a single Synapse instance at the same time.
  • Hot deployment is a feature that Synapse has been lacking for years. Being able to hot deploy a proxy service or a sequence into Synapse without having to restart the service bus is a great convenience at development time. But a configuration model based on a single XML file is not at all capable of handling such requirements.
Considering these drawbacks, I implemented a new configuration model for Synapse, known as the multi XML configuration builder. The new model, which is now the default configuration model in Synapse, loads the configuration from a structured file hierarchy, instead of loading the mediation configuration from a single XML file. With this model in place, each endpoint, sequence and proxy service has to be defined in separate XML files. The directory structure for storing these individual configuration files is as follows:
+-- registry.xml
+-- endpoints
+-- event-sources
+-- local-entries
+-- proxy-services
+-- sequences
`-- tasks
As you can see there are separate dedicated directories for each type of artifacts. Each of these directories can house zero or more XML configuration files. Each file must have the .xml extension to be recognized by the Synapse configuration builder. If you take a look at the source code you will notice that I have enforced this restriction using a Java FileFilter:
    private static FileFilter filter = new FileFilter() {
public boolean accept(File pathname) {
return (pathname.isFile() && pathname.getName().endsWith(".xml"));

private static void createProxyServices(SynapseConfiguration synapseConfig, String rootDirPath)
throws XMLStreamException {

File proxyServicesDir = new File(rootDirPath, PROXY_SERVICES_DIR);
if (proxyServicesDir.exists()) {
if (log.isDebugEnabled()) {
log.debug("Loading proxy services from : " + proxyServicesDir.getPath());
File[] proxyDefinitions = proxyServicesDir.listFiles(filter);
for (File file : proxyDefinitions) {
try {
OMElement document = parseFile(file);
ProxyService proxy = SynapseXMLConfigurationFactory.defineProxy(
synapseConfig, document);
file.getAbsolutePath(), proxy.getName());
} catch (FileNotFoundException ignored) {}
The mediation registry is defined in the registry.xml file which should be placed at the top level of the file hierarchy.
So does the multi XML configuration builder solve the problems in the old configuration model? Let’s consider the facts:
  • With Synapse configuration broken down into smaller, manageable pieces the whole configuration becomes easier to manage and keep track of. As long as the XML files are named appropriately, it is extremely easy to quickly locate a particular configuration item. We recommend using the artifact names to name the corresponding XML files. For an example the file containing the definition of the FooProxy can be named FooProxy.xml.
  • With the multi XML configuration builder, developing powerful and elegant tools for creating pieces of the service bus configuration becomes a trivial task. Also one can use conventional configuration management tools and version controlling systems such as Subversion to store and manage the configuration artifacts.
  • A team of developers can now work on configuring Synapse. Each developer in the team can work on his own configuration file or set of files.
  • Supporting hot deployment is now feasible. As a matter of fact, Ruwan implemented hot deployment and hot update support for Synapse based on the multi XML configuration builder a few weeks back. This feature is now available in the Synapse trunk and will be available for the next Synapse release.
So there you go. Target achieved. There is one little glitch in this approach though. That is how do we handle backward compatibility with older Synapse versions? For instance how can a Synapse 1.2 user, who has a single synapse.xml file, migrate to a new Synapse version? We have provided a solution for that as well. In the new Synapse configuration file hierarchy you can place a synapse.xml file at the top level (alongside with registry.xml). All the mediation components defined in this synapse.xml will be loaded to the service bus at startup along with any other components defined inside the individual directories. So a Synapse 1.2 user can simply copy the existing synapse.xml file to the synapse-config directory in a new Synapse distribution, and it will be picked up by the service bus. In addition to this convenience feature, we are planning on developing some migration tools that can help users to easily migrate an old Synapse configuration file onto a newer version of Synapse.
As far as Synapse is concerned each configuration builder should be associated with a corresponding configuration serializer implementation. Serializers are used to convert the SynapseConfiguration object model back to the textual/XML form. So for the multi XML configuration builder I developed a matching multi XML configuration serializer which can save a SynapseConfiguration object model to a file hierarchy. Similar to the configuration builder this implementation was also heavily dependent on Java file IO APIs. However, after a while we realized that the serializer is not working as expected on certain platforms; most notably on Windows. After running some debug sessions and doing some on-line reading I realized that the Java file IO operations are not consistent on every platform. As a result sometimes the serializer would encounter trouble creating a new file or moving an existing file on Windows.
At this point, my colleague, Rajika suggested using Apache Commons IO API for file manipulation. Commons IO API provides a nice layer of abstraction on top of the standard Java IO APIs. It handles all file IO operations in a consistent and platform independent manner. So I got rid of almost all the Java file IO code in the multi XML configuration serializer and replaced them with corresponding calls to the Commons IO API. Some sample code fragments are shown below:
    private void cleanUpDirectory()  throws Exception {
// If the target directory already exists and contains any files simply rename it to
// create a backup - This method does not delete the target directory
if (rootDirectory.exists() && rootDirectory.isDirectory() &&
rootDirectory.listFiles().length > 0) {

if (log.isDebugEnabled()) {
log.debug("The directory :" + rootDirectory.getPath() + " already exists. " +
"Creating a backup.");

backupDirectory = new File(rootDirectory.getParentFile(), "__tmp" +
new GregorianCalendar().getTimeInMillis());
FileUtils.moveDirectory(rootDirectory, backupDirectory);

// Create a new target directory

private void writeToFile(OMElement content, File file) throws Exception {
File tempFile = File.createTempFile("syn_mx_", ".xml");
OutputStream out = new FileOutputStream(tempFile);
XMLPrettyPrinter.prettify(content, out);

FileUtils.copyFile(tempFile, file);
FileUtils is the entry point for file IO operations in the Commons IO API. It provides you with a range of useful methods for managing and manipulating files.
Note that when writing to a file we first write the whole thing to a temporary file and once completed we copy the temp file to the final location. This was required to handle certain edge cases in the hot update implementation. Without that the hot updater will attempt to pickup and process half baked files (Again mostly on Windows – It seems Windows writes to files in chunks).
All in all, Apache Synapse is now equipped with a powerful new configuration model, a matching configuration serializer and hot deployment capabilities that leverage the new configuration model. Speaking of hot deployment, it is also based on the Axis2 hot deployment framework. Therefore it is configured in the axis2.xml file. We are yet to do a Synapse release with all these new exciting features but if you are itching to try them out feel free to grab a nightly build. Also be aware that WSO2 ESB 3.0, which is based on Synapse, has been released and that release contains all these new improvements.

See original post

Thursday, June 10, 2010

Portals and Mashups in the Cloud


Written by Tyrell

A few days ago Azeez, the senior architect behind WSO2 Stratos wrote a detailed post about its deployment architecture. He tells the story of how we migrated our carbon product stack to the cloud-native platform that we named WSO2 Stratos.

So what does the future hold for portal and mashup developers? How will Stratos change our lives?

As you can see the WSO2 Mashup Server and WSO2 Gadget Server are also deployed in Stratos as cloud-native services. This means that organisations will be able to deploy highly scalable, highly available, distributed applications with ease. And they will be far less expensive than today with a granular, pay-as-you-go model.

This is good news for mashup developers because now we can re-use code and services in a massive scale without an exponential increase in infrastructure costs as our mashups become popular. For instance, imagine that you have a mashup and plan to sell it as a service. The old model would demand that you invest heavily on servers and bandwidth, and then worry about scaling once the demand catches up. With Stratos, you can forget about this initial cost and adopt a pay-as-you-go approach. Scaling takes care of itself, leaving you time and money to invest in making your mashup awesome.

The same story applies for portals both at the portal user level and, a much more granular gadget level. Since users would want to use those super-cool gadgets from your portal in their blogs, web pages and various other places, because Stratos takes care of scalability, you will have more time to promote your portal and its gadgets . Your gadgets are using mashups? No problem.

Actually, if you carefully look at the deployment architecture, you can migrate, mashup and gadgetise any service using the set of products available there.

An example deployment of an eco-system based on WSO2 Gadget and Mashup servers will be similar to that shown below. The portal plays a prominent role helping human users collaborate. But what's noteworthy is the amount of collaboration points for non-human consumers such as third party apps via APIs and Feeds. This type of consumption is usually recursive in nature and the demand is usually unpredictable as your community grows (think Facebook, its users and the amount of Apps).

Soon, with the introduction of WSO2 ESB along with the auto scalability of Stratos, a company would be able to set up an eco-system such as the one above with minimum infrastructure costs.

See original post

Thursday, June 3, 2010

Gadgets On the Cloud


There is no doubt that JavaScript/XML gadgets make a great presentation layer over the web with increasing amount of data floating around. The ability of which these gadgets can be embedded in any place over the web, provides a great flexibility, and a wider reach. Google does this quite nicely with their iGoogle gadgets, enabling the gadgets to be embedded in almost any web page. The success of this great idea, would be only logical if all the data, services and mashups are also available over the web with open access or maybe authenticated access. This is where a cloud story fits-in, and this the very reason why Google can do it quite easily.

However, what if you want to do everything from the scratch and also provide a great presentation layer for the users. For an instance, lets say you have a lot of financial data within your enterprise, and you need to provide some of these to your customers, to general public and some for your employees. To do this, you will have to create appropriate data services, maybe mediate or transform some data, integrate with some legacy data sources, create some business work flows, mashup them with some 3rd party services like Google finance or charts and finally expose the end results to the targeted user group. This is where WSO2 Stratos PaaS comes for your rescue :)

If your requirements are such, you will need a strong middle-ware platform to full fill all the above tasks, and if its all on the cloud, you will not have to worry about anything other than writing your business logic. Once the business logic is correctly compiled, you can Mashup some of your data with external service APIs, and then write the presentation logic purely on javascript and xml as XML Gadgets and expose them to the users you need. Once the gadgets are published on WSO2 Cloud Gadget Server its just a matter of linking them up in any web page you want over the web.

Your browser does not support iframes.

Your browser does not support iframes.

The above two gadgets are taken from WSO2 Cloud Gadget Server and have linked in to this blog, to convince about the great flexibility and reach it can add-up. You do not need to use the Cloud Gadget Portal as the only place for your data to be presented (Of cause if you are not using other gadget server specific privileges such as inter-gadget communication etc). You can simply use the Gadget Server as your own gadget repository, and encourage users to discover the gadgets and embed them into their own web pages over the web.

To sum up the story I would say, try-out Stratos, try out the available services and you will definitely find out more use cases, and creative ways to use the platform and leverage the advantages of the cloud

See original post

WSO2 Stratos: WSO2 Brings The Whole SOA Stack to The Cloud


WSO2 announced that the SOA stack that they provided as downloadable packages are now available in the cloud as hosted instances with the code name WSO2 Stratos. You can try them out for free from You can register your organization for an account in the WSO2 Stratos by clicking the ‘Register’ button in the home page. You can find a detail guide on ‘How to register for WSO2 Stratos’ from Charitha’s blog,

At the registration, you will be asked to provide a username and password for the admin account. Use this credential to login as admin for the Stratos services and surf through the products. Here is a brief introduction on all the products currently available.

  • Stratos Governance: Store and govern your services, wsdls, schemas, policies and other SOA artifacts

  • Stratos Identity: Manage user bases, authentication mechanisms, permissions and all the identity aspects of your enterprise.

  • Stratos Application Server: Host your web apps, web services and manage their QoS aspects like security, reliability.

  • Stratos Gadgets Server: Write and host gadgets complaint with Google gadget standards.

  • Stratos Mashup Server: Write mashup using scripting languages like javascript.

  • Stratos Business Activity Monitor: Monitor activities of your services.

  • Stratos Enterprise Service Bus: Coming soon with message routing, intermediate message transformations, task scheduling and many more features.

With this release WSO2 bring complete SOA stack to the cloud, Now your enterprise can enjoy the power of SOA without the hassle of maintaining your own SOA infrastructure.

See original post

Wednesday, June 2, 2010

WSO2 Stratos – A true cloud story


Stratos Services

Yesterday (1st of June), A little over a 12 developer team at WSO2, took a great middle-ware platform in to the cloud. It’s not just putting all our server products on an EC2 instance, but embedding all cloud-native features into them. The PaaS (Platform as a Service) is named as WSO2 Stratos, which is based on award winning WSO2 Carbon middle-ware platform. As the Alpha-1 release, Stratos offers number of WSO2 products integrated, namely Governance Registry (GREG), Identity Server (IS), Business Activity Monitor (BAM), Mashup Server (MS) and WSO2 Gadget Server (GS).

Stratos is also offered as a downloadable version for the private cloud within your enterprise. If you are quite serious about using SOA for your enterprise and do not need to worry about deployment, scalability and server maintenance, Stratos would be the ideal solution for you.

See original post

Tuesday, June 1, 2010

The Six Weeks and 12 People Magic


Written by Sami

WSO2 Stratos Services are live, up and running.

This is the latest technology miracle from WSO2. A complete Cloud platform was built from ground up within just six weeks.

Yes we had experience with Cloud deployments thanks to governance as a service and cloud identity deployments that we did last year. But this was a novel design with an integrated view of services, with a single cloud manager and also bunch of new features.

The technology involved is bleeding edge and state of the art, with true multi-tenancy, complex tenant-aware security, billing, metering, monitoring and with six unique and comprehensive services. Simply we have built the best you can get with respect to SOA on the cloud today.

The success can be attributed to many aspects. But the paramount is the people involved. About 12 in total, and may be another 6 to 8 support roles. No more than 20 anyway. Six weeks ago, it would have been a dream, and today it is a reality. And also the vision and guidance from Paul and Sanjiva cannot be underestimated. And also the expertise of people like Azeez, Shankar, Amila and Dimuthu (and bunch of others).

Technology that we have build over the last couple of years also have helped us leap frog to this success in the cloud era. The design of the Carbon platform makes us so effective in our new designs and our ability to take up on brave new expeditions to unknown technology landscapes, and emerge successful at the end. Cloud effort has proven that the investment we have done on WSO2 Carbon was a very wise, timely and worthy. Not only that we have built technology that is unmatched by our competitors, but also WSO2 Carbon has helped us harvest a great set of skills in our team. I do not think that any company would be able to pull off this kind of a ground breaking innovative software release within this amount of time and with this amount of resources.

Personally I feel really happy, proud and content with what our team has done by today.
And as always, we have just begun. More excitement is yet to happen! There are much more to be done, and much more we are itching to achieve.

Welcome to the new era of cloud computing!!!

See original post

WSO2 Stratos Online


Written by Sami

WSO2 Cloud Services platform, WSO2 Stratos is online.

This is the latest integrated cloud middleware platform that is the most comprehensive of that kind out there.

We just started a long journey today by launching the Alpha1 version. More exciting times are ahead. Watch out the state of the art cloud platform.

See original post

Introducing WSO2 Stratos and Friends


Written by Tyrell

What's Stratos? It's the cloud-native version of WSO2's award winning Carbon Platform. I emphasised cloud-native because there's a difference between that and just deploying a web application to a service like Amazon EC2. Cloud-native implies five things ...
  1. Elasticity: Stratos manages your underlying cloud infrastructure to seamlessly handle the scalability demands of your application.
  2. Multi-tenancy: Departments, developer groups, or projects run fully independently, but share the same middleware platform for maximum resource utilization.
  3. Billing and Metering: Each tenant can meter their actual resource use for internal billing purposes.
  4. Self Provisioning: Authorized users can provision new tenants from a web portal in moments.
  5. Dynamic Discovery: Linking up services that reside in a dynamic and elastic environment can be tricky but Stratos simplifies and automates this process with standards-based service
    discovery and automatic configuration capabilities.
  6. Incremental Testing: Cloud fundamentally changes the way you test and deploy applications, but doesn't reduce your quality requirements! Stratos allows you to deploy service versions side by side and carefully dial up the traffic sent to each version.
With this initial release of Stratos, we have WSO2 Mashup Server, WSO2 Gadget Server, WSO2 Application Server, WSO2 Governance Registry, WSO2 Identity and WSO2 BAM as cloud-native instances running together seamlessly integrated. An organisation can create a tenant, activate the applications of choice and be immediately on their way.

We will be adding WSO2 ESB and other products to Stratos in the coming days. So stay tuned :)

See original post


Copyright 2009 All Rights Reserved Revolution Two Church theme modified by Milinda Pathirage