Upthrust

These days I am working on Human Task implementation for WSO2 Carbon platform, and decided to use JPA based persistence mechanism. While working on ‘Task Engine’ implementation, I found out that JPA is not just about implementing set of entities and persisting them using ‘Entity Manager’. I found out that, there are small but very important things that anyone should know to make their JPA based implementations successful. Without those you’ll stuck in the middle of some thing, deciding what is the best option. While googling I found series of articles from Vincent Partington on JPA implementation patterns which are very useful for anyone who new to JPA. In this post I am going to summarize each and every post from ‘JPA Implementation Patterns‘ article series.

Data Access Objects

This post starts with the background information about JPA and JPA implementations. After that Vincent tells us that there are more things than data access object and domain objects, like transaction handling, lazy loading and detach objects.

Then he comes to the main topic and first describes why we need DAOs(Data Access Objects) and how DAO layer which resides on top JPA can help you. After that he provides us with the information about type-safe generic DAO pattern, how to use DAO and advantages of type-safe generic DAO pattern.

In think this post is a must read for anyone who try to get started with JPA and this post help me a lot to understand issues in my JPA based implementation.

Bidirectional Associations

This post is about association between JPA entities. And it mainly describes a bidirectional association pattern used by developers who use JPA. Vincent start this post with a sample and then describes how bidirectional association can make developers life easier. After that he describes problems associated with bidirectional association and provide reader with a patter that can use to solve the issues. At last he describes the variations to the patter the introduced and what user must consider before using this pattern.

Saving (detached) Entities

This post talks about saving detached entities or merging changes done to the detached entities. First Vincent talked about how entities become detached and what  will happen when try to persist detached entity.

Then he describes about differences between Hibernate’s saveOrUpdate and JPA Entity Manager merge method.

In the last part of this post Vincent describes the problems with JPA merge and a pattern which every one can use to solve issues described with merge method.

Retrieving Entities

This post talks about two ways of retrieving an entity. One is using EntityManager.find and other is using JPQL queries. This post contains three example ways of retrieving entities by using EntityManager.find and JPQL queries.

Removing Entities

This post mainly focused on removing entities with bi-directional relationships. And Vincent has described pattern to overcome issue with removing entities when there are bi-directional associations.

Service Facades and Data Transfer Objects

This post is mainly about applying old school enterprise application architecture patterns to your JPA base implementations. First Vincent describes why we must bother about DAO’s DTO’s and Service Facades. Then he describes pros and cons of Data Transfer Objects(DTO) and pros and cons of Service Facades in JPA space. Finally he explain to you implications to your JPA architecture when you use DTO’s and Service Facades.

Lazy Loading

To use JPA to its full potential you must understand ‘Lazy Loading’ and how it works. Understanding lazy loading will allow you to utilize computing resources effectively and help you to design you entities and their relationships in way that it increase the efficiency.

First part of the post is about lazy loading and when lazy  loading occur. Then post has some information about lazy loading implementation mechanisms. After that the post describes “Runtime proxy based lazy loading” in hibernate and “Runtime bytecode instrumentation” in OpenJPA. Then there is a seperate section on differences between Hibernate and OpenJPA. Finally post contains a patter which help us to effectively use lazy loading.

PyPDF is a handy and valuable Python library for merging and splitting PDF files in Linux. It’s pure Python library built as a PDF toolkit. It is capable of:

• extracting document information (title, author, …),
• splitting documents page by page,
• merging documents page by page,
• cropping pages,
• merging multiple pages into a single page,
• encrypting and decrypting PDF files.

PyPDF is a great Python library use by many Python applications which handles PDF files directly. PDF-Shuffler is a one of the tools written based on PyPDF which you can use to merge PDF files easily in Linux. In Ubuntu you can install it using following command.

Here is a sample code that merge PDF files together using PyPDF library. In this code I have used PdfFileWriter and PdfFileReader classes from PyPDF module to read and append PDF files together. This sample doesn’t contain completed error handling logic for file handling.

You can get more understanding about usages of PyPDF if you explore more about open source projects which uses PyPDF. Here are some of the projects which use PyPDF.

• Falx: Fast, feature-rich, flexible enterprise search
• Pybrary
• xhtml2pdf-base
• nglib

Related Resources:

• Pure Python PDF to text converter
• PyPDF GUI

Support for large XML outputsData Services are Web Services that encapsulate operations on key data entities of relevance to the enterprise, thus making data integration easy for business processes, mashups, gadgets, BI application and any service in general. According to David Linthicum

Most in the SOA community understand that data services provide controlled interfaces to underlying data, but typically don’t understand the strategic value of data services to the SOA.

He emphasized the fact that those who focus on the notion of a service as delivering functional behavior, neglect the need to manage the underlying data. He tells that

In many cases, data quality and consistency issues quickly arise, and the agility that SOA should provide is limited by the need to alter services directly after the underlying data has changed.

David argues that data services created and leveraged correctly within the context of a SOA, should provide a wide variety of features including data quality assurance, data governance, and, most importantly, the ability to support data abstractions.

Even though there are lot of benefits(David’s opinion is further elaborated by Ash Parikh), getting data service right is not an easy task. You need to consider about the design, implementation, technologies and tools you are going to use for this. You can find several guide lines and best practices in designing and implementing data services from articles “Introduction to Data Services“, “Incorporating Enterprise Data into SOA” and “How To Get Started with Data-Orientation – What Architects Told Me…“.

Even you get the concepts and your designs correct, you must find a tool which allows you to incorporate you design and best practices into the implementation in flexible and easy manner. There are so many tools out there which provide support for developing and deploying data services written following various standards. In this post I am going to introduce you to the WSO2 Data Services Server, which is a award winning product from WSO2.

WSO2’s award winning Data Services solution WSO2 Data Services Server is a data services solution that will help you to achieve your targets in data orientation.

With the recent 2.2 release WSO2 Data Services Server provides some major improvements to it’s industry leading data services solution. Following new features are included in this release.

• Support for large XML outputs – The core engine has been modified to support XML streaming. This has resulted in two notable improvements.
  • Efficient use of server memory – No matter how large the payload is, server memory does not grow propotional to it. The streaming capabilies push data to client side as and when needed
  • Improved response time
• Google Spreadsheets as a data source
• Content filtering based on user roles
• Support for named parameters
• Ability to configure schema type for output elements
• Mixing multiple data source types in nested queries
• Excel 2007 support
• Support for Oracle Ref Cursor – Oracle Ref cursor is a Data type. A variable created using this data type is usually called as a Cursor Variable. Some of the primary advantages of using a ref cursor are,
  • ability to pass resultset between sub programs (eg: functions, stored procedures)
  • dynamic queries
  • efficient memory utilization

For more tutorials on WSO2 Data Services Server visit wso2.org.

Sri Lanka President Mahinda Rajapaksa yesterday honored three outstanding athletes of the country in a ceremony held at the Temple Trees.

President Rajapaksa presented awards to Sprinter Susanthika Jayasinghe and cricket legends Sanath Jayasuriya and Muttiah Muralitharan for their exemplary contribution to the country.

Susanthika Jayasinghe, 34, is the only Sri Lankan athlete to win an Olympic medal. She won a bronze in women’s 200m at the Sydney Olympics in 2000 but later awarded the silver after Marion Jones was stripped off the gold medal due to drug use.

Muttiah Muralitharan, 37, is the world’s highest wicket taker in both Test cricket and One-Day Internationals. He has been playing since 1992.

These people make us proud. They toiled for their country & brought us recognition & high regard in the world. They did not go around the world to destroy our country with lies. They did not betray our country. They did not try to hog the fame all for themselves. They were team players & acknowledged humbly the help & contribution of others. They did not ask for perks, for more & more & even more. To paraphrase, in the words of President John F. Kennedy – “They did not ask what their country could do for them, but went ahead & did what they could do for their country”. They silently & diligently went about their task & achieved success not only for themselves but for the whole country. Sanath, Muttiah, Susanthika – You are our heroes & today you are being felicitated by our National Saviour & Hero of the century, H.E. The President, Mahinda Rajapaksa. This is truly, an historic occasion, when the good people, who have worked for the welfare of our country meet. We, the people of Sri Lanka honour you all with deep affection & love for what you have so nobly done for our motherland & the history of our nation is so much the greater by being adorned by your achievements. May the Blessings of the Triple Gem, be upon you all always.

-  Comment by Sunil Jayanth Weliwitigoda in Adaderana.lk

Sources

• http://www.adaderana.lk/news.php?mode=head&nid=65
• http://www.colombopage.com/archive_10/Jan1262438623CH.html

Adobe AIR is a cross-operating system runtime that lets developers combine HTML, Ajax, Adobe Flash, and Adobe Flex technologies to deploy rich Internet applications (RIAs) on the desktop. The most recent version is Adobe Air 2 with more features than ever for the users as well as developers.

AIR 2 builds on the success of AIR 1 by giving developers new capabilities, and even tighter integration with the desktop. Some new features of AIR 2 include:

• Support for the detection of mass storage devices.
• Advanced networking capabilities like secure sockets, UDP support, and the ability to listen on sockets.
• Support for native code integration.
• The ability to open a file with its default application.
• Multi-touch and gesture support.
• New APIs for access to raw microphone data.
• Webkit update with HTML5/CSS3 support.
• Global error handling.
• Improved cross-platform printing
• Improved security and support for enterprise and government standards.

The beta release of Adobe Air 2 is available for download from Adobe site. But the problem is they don’t provide us with a 64-bit Linux version(for Ubuntu 9.10 64-bit). But there are some workarounds to this problem. You can install 32-bit version in 64-bit Ubuntu by installing some 32-bit libraries.

First, install ‘getlibs‘ tool from http://frozenfox.freehostia.com/cappy/getlibs-all.deb. Then install ‘libhal-storage.so.1′ library using following command.

After the above step please download the Adobe Air 2 beta version’s Linux *.bin file from Adobe, and install it by making it a executable and run the bin file in command line.

If every things went well, now you have Adobe Air 2 beta installed on your system. Please restart your browsers and try to navigate to some Adobe Air application sites and install your favorite Air applications from them.

Please note that I have ‘ia32-libs‘ installed in my Ubuntu 9.10 previously. If you have any problem running Adobe Air using above two steps, try to install the ‘ia32-libs’ using Ubuntu package manager. Also refer this knowledge base article from Adobe on installing Adobe Air 1.5 in 64-bit Linux including Ubuntu.

For more information Adobe Air 2:

• Introducing Adobe AIR 2 beta
• What’s Coming in Adobe AIR 2

Utility applications in Linux which are(most of them) originally created for Unix and ported to Linux can be used to make every Linux users life easier. It doesn’t matter whether you are a beginner or a hard core Linux geek, if you know the tools it’ll save you lot of time.

In this post I am going to give you introduction to several utilities available on Linux that originated from Unix based on the recent posts by Peteris Krumins. I am planning to update this post time to time once he add new articles about more tools.

Pipe Viewer

Pipe viewer (Written by Andrew Wood) or pv in short can be used inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion.

Default Ubuntu installation doesn’t come with this tool. You need to install it using ’sudo apt-get install pv’ command.

To get start with pv, lets use this to monitor the progress of compressing large file containing giga bytes of information in to a small file. The normal way to do this using gzip is like following.

But this command won’t tell you how much time it takes to compress this file or monitor the progress of compression.

By using pv you can precisely time how long it will take. Take a look at doing the same through pv:

Pipe viewer acts as “cat” here, except it also adds a progress bar. We can see that gzip processed 69MB of data in 4 seconds. It has processed 74% of all data and it will take 1 more seconds to finish.

There are several advance usage patterns of pv command and I am not going to cover those.Please refer the article from Peteris to explore more about pv command.

lsof(list open files)

Peteris called this tools as the Swiss Army Knife of Unix Debugging. lsof is a utility command that you can used to list information about files opened by  Unix/Linux processes. In Unix/Linux every things is a file: pipes are files, IP sockets are files, unix sockets are files, directories are files, devices are files, inodes are files… So you know the advantage of this kind of tools where every thing is a file.

Using lsof

Here are some common uage scenarios of lsof comand extracted from catonmat.net blog. You must have root permission to get the information about all the open files using lsof. Unless otherwise you’ll only get information set of files which you have permission to access them.

List all open files

Running lsof without any arguments lists all open files by all processes.

Find who’s using a file

With an argument of a path to a file, lsof lists all the processes, which are using the file in some way.

You may also specify several files, which lists all the processes, which are using all the files:

Find all open files in a directory recursively

With the 

argument lsof finds all files in the specified directory and all the subdirectories.

There more use cases of lsof command like getting open files by process, open files by users, list all network connections, list all TCP connections, list network activity by user and etc. Please refer blog post from Peteris for those usage scenarios. You can find some examples of lsof from this link also.

SOA is not only about service hosting(implementing Web Services that expose certain business functionality over network). It’s about enhancing efficiency, agility and productivity of an enterprise by applying service oriented design paradigm and service oriented architecture concepts to when implementing information systems which are critical to business functionality of enterprises.

So when it comes to implementing SOAs, there is a need for several technologies, products, APIs, supporting infrastructure and various other parts. In a typical SOA implementation at least there will be a service hosting environment, an ESB, a service governance tool, a business process management system, a portal server and set of monitoring tool which used to monitor the whole system real-time for fine tuning business processes, service APIs and etc.. There are lot of tools and products available which provide the above mentioned functionalities. Some provide set of functionalities and some has several products that provides most of the functionalities separately. The problems with those existing products and platforms are they are not very well fit together even though they are from same vendor or most of the time those platforms are heavy weight and required lot of time to getting started with them and getting familiar with the tools.

For a long time IT departments heavily suffered from drawbacks of the available products. The cost, complexity, time need to getting familiar with the products and the maintenance issues became a real pain to the enterprises use these products.

With the recent announcements on availability of WSO2 Gadget Server and WSO2 Business Activity Monitor, WSO2 SOA Platform has become the most complete and comprehensive open source SOA platform which provides all the functionalities required to full fill SOA implementors requirements while keeping it’s simplicity intact.

I can assure you that it is the most light weight, high performance, extensible and most user friendly SOA platform out there. Because of the power of OSGi based WSO2 Carbon platform, we were able to build unified, consistent and fully componentized SOA platform out there in the market.

If you want to know how WSO2 come here to the state where we are now, read the recent blog post from Samisa, director of engineering WSO2. Also this not the end, this is only the start of becoming world’s best SOA platform provider.

Information systems and enterprise application have fundamental assets of companies and they rely on them to perform business operations. Information systems can help organizations to improve efficiency and reduce costs by automation of business processes. But in reality these applications should align with business processes with the change of market conditions and flexibility and agility of enterprise information systems are become more important than ever.

Service oriented architectures (SOA) are currently a hot topic and touted as a key to business agility and solution to a most problems in enterprise information systems. But the success of SOA is totaly depending upon the alignment of purpose and objectives between IT and business. In their latest article, IBM’s Jens Andexer and Willem Bekker from Standard Bank provide some sample of the good, the bad and the ugly business aspects of SOA.

They have divided impact on business by SOA into several categories and described the pros and cons of each.

Agility

• The good: Two main concepts behind SOA are reuse and loose-coupling. By reusing existing applications you can create new applications. And by applying loose-coupling you can minimize the impact of change, and therefore it’s easier to adopt your software to changing requirements. So SOA allows organizations to reduce time to time to market and adaptability and this will increase competitiveness of your organization.
• The bad: Requirement of introduction of new entity(Center of Excellence) which provides technical expertise to the rest of the organization can leads to conflicts among the different sections of the organization.
• The ugly: Transforming organization to become service oriented can be complex and expensive. In addition to that there is no shortage of opponents to the change.

Alignment

• The good: Alignment of IT functionality to business functionality and describing IT functionality in business terms, SOA facilitates a closer collaborative relationship between business and IT.
• The bad: Placing the ownership and control of services into the domain of business changes the power structure in organizations. This is typically met with resistance from those who have a vested interest in keeping the status quo.
• The ugly: Implementing SOA in your organization not only require change in technology aspects, it also require change in organization structure and culture. The organization must learn what is meant by agility and how to effectively use SOA. the ugly truth is that this is the one of the difficult lesson to learn.

Business Process Improvements

• The good: When applying SOA to your organization, it involves business process re-engineering. This will help organizations to improve their existing business processes.
• The bad: When implementing SOA, business must involve in defining services and designing of the IT systems. This is not the typical role played by the businesses and it’ll become uncomfortable change.

Flexibility

• The good: Good software engineering practices enforced by SOA will make IT to be more responsive to the changing business needs.
• The bad: Introduction of services make complexities of the IT a secret. But on the other hand SOA implementations typically depends on the set o technologies like business process execution engines, ESBs and etc.
• The ugly: An SOA initiative is founded on the promise of delivering business value quicker and cheaper than before. But SOA that is too technology focused is unlikely to deliver on that promise since they will not show value in terms business people want to see it.

Data Unification

• The good: Interoperability is a aspect of SOA. The design of inter-operable service interfaces provides an opportunity for data unification across enterprise IT systems. Unified here means common:
  • Structure – the structural relationships between elements is the same
  • Semantics – semantic refers to the meaning and use of the data. Data must have a consistent meaning and must not be used in a way that can be misleading
  • Format – how data is represented is important
  • Type – A type is determined by the representation of data and the set of behavior that can be performed on it
  • Timing – Timing refers to when an attribute is updated
  • Life cycle – under what circumstances data is add to data bases and when it is updated and when and how it is finally deleted from data bases
• The bad: Typically this type of data unification doesn’t exist. Trying to develop it often shows how disparate views in the organization may be.
• The ugly: It hard to address all the characteristics of consistent data and the ugly truth is that unifor service interface is very difficult to build.

Operational Monitoring

• The good: The technologies and principles used to enable a SOA make monitoring business process easier. This allows organizations quickly adopt their systems to meet their business goals.
• The bad: But building a system that can actually provide valuable outcome requires specialist skills.

Leveraging Operational Systems

• The good: In most scenarios SOA use existing systems to supply the business functionality to services. This will reduce the costs while allowing organizations to use their money for further improvements in their businesses.
• The bad: In some cases it’s hard to repackage existing systems in to services.
• The ugly: In some cases operational systems may need to be changed or additional logic/implementations might be required.

Summary

“For SOA to be a success, business and IT must align there purpose and objectives. IT driven SOAs fail because they are perceived as a change in technology that has no direct benefit to business. The case for SOA must strengthen the organization’s existing goals and strategy and this goes beyond an IT centred approach. This represents a culture change to where business expectations drive the priorities for IT which when coupled with a culture of co-operation between business and IT lays the ground work for success.”

Via InfoQ.

WSO2 Business Process Server team released the latest version(1.1.0) of WSO2 BPS yesterday 19th November 2009.  As the successor of WSO2 BPS 1.0.1, WSO2 BPS 1.1.0 comes with handful of new features which will make process developer’s life easier.

WSO2 BPS 1.1.0 uses revision of Apache ODE trunk which is going to be the future of Apache ODE BPEL engine as it BPEL engine. In addition to that there are several bug fixes to the Apache ODE like fixing E4X extension support when using OpenJPA data access objects. WSO2 BPS 1.1.0 comes with a completely new integration layer for Apache ODE which integrated ODE with WSO2 Carbon platform. Now the previously unsecured BPEL management API is completely secured based on WSO2 Carbon authentication mechanism and new version has a completely re-written Web service API for managing your BPEL engine. There are some improvements to be done on the management API and those will be available with the upcoming releases.

By leveraging the extension mechanism of new ODE engine, now you can extend WSO2 BPS’s BPEL language by introducing your own extension to BPEL language and deploying them on WSO2 BPS.

This version of WSO2 BPS has the provisioning support based in Equinox P2; now users can extend the BPS by installing features available in WSO2 Carbon P2 repositories. For example by installing mediation feature, you’ll be able to do service mediation using WSO2 BPS.

Here is the list of new features available in WSO2 BPS 1.1.0

• Apache ODE trunk’s revision is used as the BPEL Engine
• New WSO2 Carbon integration layer for Apache ODE
• BPEL Process Versioning Support
• Support for invoking secured(Using WS-Security) partner services.
• Experimental clustering support
• Invoke activity recovery support through management console
• E4X based data manipulation support for BPEL assignments
• BPEL Extension support for extending WS-BPEL language
• Equinox P2 based provisioning support –  extend your BPS instance by installing new P2 features [5].

You can download this release at http://wso2.org/downloads/bps.