Discussing the process of integration

Phasing out of this blog.

2011-10-18T10:39:00.001-07:00

Hi all. I decided to phase out this blog for various reasons and create a new blog with fresh perspective of my various current interests. Please visit my new blog site.

Top 10 Approaches for Professional Development

2010-10-25T16:06:00.000-07:00

I found this article in February 2010 issue of "HIMSS Clinical Informatics Insights" magazine.

I think that it works in any sector and at any level. You might be doing some of these already, but this article shows what else you can do to successfully grow professionally in any job you choose. Infact, I am testing those rules on my career too. I will let you know how succesful I am in few months time.

Also, don't forget to read my other article - Healthcare IT jobs... What should you know?.

- Cheers.

Delighted to join HIMSS Southern California chapter

2010-03-11T10:52:00.000-08:00

I am delighted to join Southern California's HIMSS(Healthcare Information and Management Systems Society) chapter. It is a brand new start for me in healthcare IT. I am hoping to attend the meetings and do some volunteer work.Here is my membership card.

Healthcare IT books

2010-01-03T10:45:00.000-08:00

Here is a link for you, in case you are searching for a "Healthcare IT book store".

Choosing to solve the right Healthcare IT problem first.

2009-12-07T15:59:00.000-08:00

I found another nice article in the "The Healthcare IT guy" blog. Instead of rephrasing it, I am just linking to the blog posting.

You must read it if you are venturing into "Healthcare IT solutions".

Healthcare IT jobs... What should you know?

2009-12-07T15:29:00.000-08:00

I recently got very curious about this whole Healthcare sector, and started doing some research about it's current needs and the kind of IT people who fit into this sector. As I was looking around, I came across a nice blog posting from Shahid Shah.

"...

If you’re not an expert in healthcare IT, how could enter the field? It’s actually a little easier than you might think but you’ll need to be creative. Here are some ideas.

If you’ve got experience running or working in a medical office or you’re an experienced project manager you can apply for an implementation specialist or assistant at almost any healthcare IT firm like an EMR or EHR vendor, consulting firm, or systems integrator. The thing to keep in mind is that every customer that buys an EMR needs to have it installed and deployed and that’s done by implementation folks. There is a shortage of people that can take complex products like EHRs and EMRs live.

If you have a little or a lot of general IT experience but no healthcare IT experience you can start by working in a technical support or training capacity. You would get the opportunity to learn new products and use your IT experience to provide customer service, support, and training talent.

If you’re interested in the software side you can think of being a tester of software; vendors need good quality assurance and configuration management personnel and that’s a great place to begin your healthcare IT career.

If you’re good at writing, consider joining the documentation team for creating training materials, videos, screencasts, or other related artifacts necessary to teach people how to use healthcare IT.

If you’re a developer interested in writing software but you’re not experienced in healthcare, join one of the many open source projects that are out there building open source EMRs, EHRs, PHRs, and related tools. Open source is a great way to join a community of people willing to help you if you’re willing to give back to them, too.

If you’re an integration specialist (you know EAI, EDI, EII, ETL, ESBs, or other integration techniques) start to learn HL7, CCR, and CCD and you can write your own ticket almost anywhere. The majority of healthcare problems in the IT arena are integration and deployment problems so if you know scripting and HL7 you’re good to go.

..."

Integration Concerns for Healthcare Organizations

2009-11-30T16:25:00.000-08:00

"...

Today’s healthcare organizations face many challenges in implementing, managing and maintaining HL7 integration solutions. Integration solutions must be able to support mission-critical healthcare systems, and meet the requirements of the current regional and provincial initiatives while at the same time ensuring future EHR initiatives can be accommodated within the solution’s framework.
This event will focus on tools, methodologies and best practices to help simplify and accelerate Healthcare Integration solutions. Attend and learn from real-world success stories on how Microsoft-based technologies were applied to deliver healthcare solutions.

- EHR and HealthVault
- HL7 v2 and v3
- Regional Integration
- Patient & Provider Portals
- Paperless Clinical Operation
- Medical Device Integration
- Reporting & Analytics
- Performance Management

..."

Watch this Web cast.

Microsoft ESB Guidence/Toolkit Videos

2009-11-30T15:08:00.000-08:00

Peter Kelcey's 7 part Video series
Brian Loesgen's 2 ESB demos
From Microsoft ESB site
An ESB itinerary introduction demo

BAM Alerts issue while installing BizTalk Server 2009

2009-07-01T15:03:00.000-07:00

Problem:

While configuring BizTalk Server's BAM Alerts, you might get the following error:

Microsoft (R) Business Activity Monitoring Utility Version 3.8.368.0
Copyright (C) 2006 Microsoft Corporation. All rights reserved.
ERROR: Failed to set up BAM database(s).
There was a failure while executing nscontrol.exe. Error:"Microsoft Notification Services Control Utility 9.0.242.0
Microsoft Corp. All rights reserved.

Analysis:

This happens when the notification services are not properly installed in this environment. The real problem is that – Notification services is not part of the SQL Server 2008 installation, unlike SQL Server 2005.

http://sqlblog.com/blogs/tibor_karaszi/archive/2007/08/03/no-notification-services-in-sql-server-2008.aspx

http://social.msdn.microsoft.com/Forums/en-US/sqlnotificationservices/thread/03cb63a9-7fe7-408d-b194-7ff3d14e682a

So, products like BizTalk will eventually rely on a different mechanism to notify ( for alerts). I also found few blog postings which might of some help to you.

http://social.msdn.microsoft.com/Forums/en-US/biztalkgeneral/thread/7f4c1060-c47c-4467-8ada-778bb47f7c1c

http://msinnovations.spaces.live.com/blog/cns!62E68922E47BC425!609.entry

The person who posted this could finally make the installation work by taking the following steps:

“VM Base: Windows Server 2003 R2 SP2, SQL 2008, VS 2008 (minus SQL Express) installed in this order.
VM Diff:
• Don't use the SQL 2005 SSNS RC1 linked in the 2009 beta install guide. Use a SQL 2005 full version.
• Install SQL 2005 as a named instance (installing after SQL 2008 was the default instance worked fine for me), install the database services, notification services, and analysis services.
• Run SQL 2005 SP2. If you forget this step, the BAM Alerts tells you the current SQL 2005 instance is not supported.
• Install BizTalk 2009 full
• Configure BizTalk 2009, use the named instance of SQL 2005 for all of the BAM databases. “

Another way could be –

“I have been able to get SQL Notifications working with Biztalk 2009 and SQL Server 2008. In the installation doc for BizTalk 2009 under the "Install SQL Notification Services" section there are a couple of hotfixs to download. SQLServer2005_XMO and Yukon_SP2_CU9_SNAC. You receive an email with the download link and a password. The Install doc say to leave the password blank. Enter the password and an msi will be extracted to your directory. Install the msi and for each of the 2 downloads. Extract and install the Yukon_SP2_CU9_SNAC first.

Once this section of the doc is complete you can now configure the BAM Alerts (Which will create the Databases. You will have to ensure that your BAM users (BAM Notifications Services User, BAM Management Web Servcies User) have the correct access to the database tables. Check out this link for windows groups and user accounts (http://msdn.microsoft.com/en-us/library/aa577661.aspx)”

My suggestion:

“BAM Alerts “ are not the most critical for BizTalk’s primary function. It is just an add on feature that most of the clients do not use. Bam Alerts basically use SQL Server’s Notification services to send alerts to BizTalk Notification subscribers. So far, I have not seen this feature being used by any of the clients I have worked with.

BizTalk 2009 books in the market

2009-04-27T11:31:00.000-07:00

Hey BizTalkers, you must be all excited about the new version of BizTalk 2009 with lot of cool features, and must be waiting to get your hands dirty. Here are the books that help you learn the new features and set you up for your dirty work. :)

The authors are almost done with most of their content, and some are in the market already.

BizTalk Server 2009 Roadmap

2008-09-23T10:32:00.000-07:00

Finally, the roadmap for BizTalk Server 2009 is revealed. It appears to have lot of new and exciting features in it. I can't wait to play with it.

Updated list of BizTalk 2006 R2 posters

2008-09-23T10:27:00.000-07:00

Here is the updated list of BizTalk 2006 R2 posters. It also has interactive capabilities reader. Enjoy.

Microsoft's Messaging, Workflow Roadmap Announced

2008-04-21T16:00:00.000-07:00

" A project code-named Oslo will deliver updated messaging and workflow
technologies in the next version of BizTalk Server and other products
starting in 2009." - Research Directions

Here is the roadmap

Debatching XML using Index Functoid and Map parameters

2008-04-15T16:26:00.000-07:00

Scenario:
In one of the applications that I am actively working on has need to de-batch a Product from an XML which has "n" number of products in it, and persist one product at a time by calling a stored procedure. I have to loop through the collection of products and persist individual Product. Just to make it worse, each of the stored procedure calls need more than the information provided in the Product node.

Solution:
There are many ways of retrieving the product information from the XML, like XPaths and loops. But, in this scenario I wanted to try something different. I decided to use a combination of - Index Functoids, Parameters to maps, and looping of Products. I did the following to accomplish this task -

a. I created a schema which holds the index of the loop, and called it context schema. A new message of this type will be created in every iteration of the loop.

b. I created a interesting multi-part message map. It has 2 input schemas - the order schema with multiple products in it; the second one is the above context schema. The interesting thing here is that, I can pass the index of the loop into the map and get the correct product out of the product collection, and pass the values to the target SQL request schema.

c. A simple Orchestration does the rest of the work. It looks like below -

If you look at it closely it just does the following to accomplish the task:

1. Get Item Record Count Expression shape - gets the count of Products in the order message.

2. while the there are products continue the loop.

3. Construct a context message that will hold the current index value of the loop. This message will be used as a "parameter to the map".

4. Create a map which

a. takes two input messages

b. and gives one message which will be used in calling the stored procedure

5. The stored procedure is now called with the request and a response comes back from it.

6. Finally, the loop is incremented to pick the next Product from the Order message.

That's it. Now you know how easy it is to use Index functoid and multi-part maps to get the specific information from a record.

ESB Evaluation Framework

2008-04-15T15:14:00.000-07:00

I accidentally came across the "Enterprise Service Bus Evaluation Framework" by Patricia Seybold Group. The author of this article Brenda M. Michelson identified the very improtant criteria in evaluating the Enterprise Service Bus.

- Integration Scenarios
- Design, Development and Deployment
- Management and monitoring
- Architecture
- Product Viability
- Company Viability

I suggest you to consider these in evaluating your next ESB product for your organization.

BizTalk magazine - BizTalk Hot rod Issue3

2008-04-07T15:50:00.000-07:00

I recently came across a BizTalk magazine, named BizTalk Hotrod. Infact, this is the only BizTalk maganize dedicated purely to BizTalk that I found till this point. It has lot of interesting articles from lot of BizTalk specialists. Sal & Todd put together a great magazine. They recently released the Issue 3 of Hot rod magazine. You can also download the PDF version of the magazine.

WCF primer

2008-02-17T08:59:00.000-08:00

I am not a man of 1000 words. Instead, I like being very visual. And it makes me understand the concpets better. I am pretty sure that, I am not the only .NET developer looking for .NET 3.5 stuff, WCF in specific.So, the attached a set of Visio diagrams which made it easy for me put concepts in context. I suggest you to download and browse through it for a better understanding of WCF. And, Happy coding ...

1. Server Side WCF

2. CLient side WCF

3. WCF Framework

BizTalk Case study – A Ticketing System (Part 1)

2007-08-17T23:22:00.000-07:00

Introduction:

Couple years ago, I was taken as consultant to solve a business problem of automating a concert ticketing process. During the implementation of this application I learnt lot good things about asynchronous communication, ordered processing and reliability. In this article, I will take you through the entire project life cycle of this application. During this I will try to avoid non relevant details and stick to the usage of BizTalk and web services; and make it as interesting as possible. I will use a fictitious ticketing web application named mytickets.com in the article.

Overview:

Mytickets.com is nation-wide Rock concert ticket selling company. It caters it’s customers by selling concert tickets through it’s ticketing web application. Like any reservation system, it has a internet web application for customers who go through a series of steps before they confirm their ticket.

Usage scenario:

The above scenario is a very common one. But, what happens underneath in the system is complicated. To avoid the unnecessary details I will walk you through the following critical usage scenario of the Ticketing system:
- System shows the concert listing and inventory to the customer.
- Customer selects the tickets for a specific concert.
- Customer Logs into his/her profile.
- Customer puts credit card information, and personal billing information
- System sends an immediate conformation which says – “ Your request
is under consideration, here is your conformation number, and we will send an email with the tickets in few minutes.”
- At some point later user gets another email with either confirmation or
Failure notification (Email).
Let’s delve into the details of this.

1. Generation of Confirmation number & Priority number

System has to generate a confirmation number and a priority number for every customer request. Confirmation number will be used by the customer to track his request with mytickets.com, incase the second email(ticket Confirmation email) does not arrive on time, and also for any other clarifications. Priority number tells the customer that his ticket is being processed in the order in which the requests have arrived, and his request is located at the n th location in the queue. For example, he is the 16th person to request for the concert. This will also protect mytickets.com from any law suites by customers for out of order processing of requests. Ticketing system uses an internal web service to create a confirmation and priority numbers.

2. Sending first E-mail to the customer

Just after receiving the confirmation and priority numbers an email is sent to the client that says - “ Your request is under consideration, here is your conformation number and priority number, and we will send an email with the tickets in few minutes.” This e-mail is sent by the ticketing web application. SMTP service of IIS server can send the email to the customer. (Refer to Previous Sequence Diagram)

3. Processing the request and sending the confirmation e-mail to the customer

The customer’s ticket request is processed in the order arrival. His credit card and address are verified. If the initial validation fails then, a failure e-mail is sent to him immediately. Otherwise, after the initial validation, availability of seats for the selected concert is verified. If the seats are available then the inventory is updated and a confirmation email is sent to the user. This process is complicated and also the system needs to process the request orderly fashion based on priority number. The design of the process also needs to consider the high volume scenario. (Consider buying tickets for “Rolling Stones” Concert) and reliability. So BizTalk is used to take care of the purchase process.

The architecture of the ticketing application looks as below:

In the part 2 of this posting I will walk you through the implementation of this system.

BizTalk Tools and guidelines

2007-08-16T10:22:00.000-07:00

I have been collecting the tools and guidelines coded and written by BizTalk developers and designers. I always wanted share these with the developer community. I finally put all of them in one place, and also classified them based on the life cycle stage of the BizTalk project. - Happy coding.

Design:

1. Naming Conventions(Author: Scott Colestock)
2. Enterprise Integration Patterns(Author:Gregor Hohpe)
3. Pattern Wizard(Author/Company: Codeplex.com,BTS Version:2006)
4. Orchestration Designer for Business Analysts(Author/Company:Microsoft,BTS Version:2006)

Development:

1. Pipeline Viewer(Author:Jon Flander)
2. SSO Config App(Author:Jon Flander)
3. Context Adder(Author:Jon Flander)
4. Schema Viewer(Author:Jon Flander)
5. Correlation with MSMQ(Author:Jon Flander)
6. OrderMapping.exe(Author/Company:Microsoft)
7. ExportCatalog.exe(Author/Company:Microsoft)
8. Std. BizTalk Tools(Author/Company:Microsoft,BTS Version:2006)
9. DebugView for Windows v4.64 (Author/Company:Mark Russinovich, Microsoft;BTS Version:2006)
10. TCP Trace(Author/Company:PocketSOAP,BTS Version:2006)
11. Delimited File schema Generator(Author/Company:gotdotnet.com,BTS Version:2004)
12. Adapter Wizard(Author/Company:codeplex.com,BTS Version:2006)
13. Pipeline testing Tools(Author/Company: Microsoft BizTalk Std SDK tool,BTS Version:2006)
14. Best Practices Analyzer(Author/Company:Microsoft,BTS Version:2006)
15. Understanding the BizTalk Server WS-Addressing Helper(Author/Company:Microsoft,BTS Version:2006)
16. Rule Manager(Author/Company: Acuman Business,BTS Version: 2006)
17. BizTalk Server Pipeline Component Wizard(Author/Company: Codeplex.com,BTS Version: 2006)
18. BizTalk Server Orchestration profiler(Author/Company: Codeplex.com<,BTS Version: 2006)
19. WCF Adapter for BizTalk server 2006(Author/Company: Codeplex.com,BTS Version: 2006)
20. TCP/IP adapter for BizTalk server 2006(Author/Company: Codeplex.com,BTS Version: 2006)
21. BizTalk Software factory(Author/Company: Codeplex.com,BTS Version: 2006)
22. Project Solution Kits(Author/Company: Codeplex.com,BTS Version: 2006)
23. Map Cop(Author/Company: Codeplex.com,BTS Version: 2006)
24. ADO Adapter for BizTalk server(Author/Company: Codeplex.com,BTS Version: 2006)
25. XChain(Author/Company: Codeplex.com,BTS Version: 2006)
26. Debatching sample for BizTalk server 2004 and SQL Adapter(Author/Company: Codeplex.com,BTS Version: 2006)
27. Custom Event log Functoid(Author/Company: Codeplex.com,BTS Version: 2006)
28. Custom Receive File Adapter(Author/Company: Codeplex.com,BTS Version: 2006)
29. Tool for generating metadata(Author/Company:Jesus Rodriguez,BTS Version: 2006)
30. TIBCO Adapter(Author/Company:Pallavi Narayanaswamy, gotdotnet.com;BTS Version: 2004)
31. Batch Execution Framework for BizTalk 2004 Orchestrations(Author/Company: Pallavi Narayanaswamy, gotdotnet.com;BTS Version: 2004)

Testing:

1. BizUnit(Author/Company: Codeplex.com,BTS Version: 2006)
2. BizUnit Extensions(Author/Company: Codeplex.com,BTS Version: 2006)
3. LoadGen Tool(Author/Company: Microsoft,BTS Version: 2004)
4. Quick Counters(Author/Company:Codeplex.com,BTS Version: 2006)
5. PerfMon counters(Windows OS: $SystemRoot\system32\perfmon.msc,BTS Version:2004/2006)

Configuration/Management:

1. BizTalk Assembly Viewer(Author/Company: Microsoft BizTalk Std SDK tool,BTS Version:2004/2006)
2. BizTalk Subscription Viewer(Author/Company: Microsoft BizTalk Std SDK,BTS Version:2004/2006)
3. Assembly checker and remote GAC Tool(Author/Company:Codeplex.com,BTS Version: 2006)
4. BizTalk Host Configurator per Application(Author/Company: Codeplex.com,BTS Version: 2006)
5. Storing Usernames, Passwords in SSODB Database using MMC 3.0-Biztalk(Author/Company: Codeplex.com, BTS Version: 2006)
6. BizTalk Service instance controller(Author/Company: Patrick Wellink,BTS Version: 2006)
7. SDC tasks library(Author/Company: Codeplex.com,BTS Version: 2006)

Deployment:

1. Deployment Frameworks for BizTalk 2004/2006(Author: Scott Colestock,BTS Version: 2004/2006)
2. Deploy Tool(Author: Richard<,BTS Version: 2006)
3. Log4Net for BizTalk(Author: Scott Colestock<,BTS Version: 2004/2006)
4. BizTalk Namespace verification tool(Author: Jon Flander,BTS Version: 2006)

Misc:

1. System Verification tool v1.0(Author/Company: Microsoft,BTS Version: 2006)
2. Restore Script for BizTalk 2006 Std. Edition(Author/Company: Microsoft ,BTS Version: 2006)
3. BizTalk 2006 Documenter(Author/Company: Codeplex.com,BTS Version: 2006)
4. Powershell BizTalk Provider(Author/Company:Codeplex.com,BTS Version: 2006)
5. File Dump Utility(Author/Company: gotdotnet.com,BTS Version: 2004)
6. SSO Trace Utility(Author/Company: Microsoft,BTS Version: 2006)

Binding of orchestration's logical ports

2007-07-21T16:51:00.000-07:00

Binding of orchestration's logical ports is one of the most basic features that every BizTalk developer uses. But, it also confuses lot of BizTalk newbies. Being a developer myself I understand the pain. Initally, when I started coding BizTalk, I, myself looked for a very simple working code which covers some of the scenarios.

During my "googling" process I found many good references. Some of them went deep into the thoeretical side of the port bindings, and some of them covered very specific scenarios. But I could not find any code samples which do a comparision of diffrent features of orchestration's logical port bindings. So, I thought, I would help the BizTalk newbies in understanding logical port bindings, by putting together some very simple code samples. And the resultant code covered the following scenarios:

1. Direct port binding
      a. Direct binding with "Partner ports" (Multiple forward partners scenario)
      b. Direct binding with "MessageBox"
      c. Direct Binding with "Self correlating ports"
2. Dynamic port binding
3. Specify Later Port Bindings
4. Specify Now Port Binding

And this code can downloaded from my code samples page(It's called "BizTalk Development - Orchestration Port Bindings").

Thomas Erl and his contributions to SOA

2007-07-18T14:09:00.000-07:00

You must have heard the names Thomas Erl and SOA together many times. He is one of leading authors of SOA based books and articles, and gave lot of presentations on SOA. I read two of his three books on SOA. And I am a big fan of his contributions to SOA world. Just in case you did not hear the name; the following links will help you so.

- His web site
- His SOA Books
- His SOA Papers
- Editor of SOA Magazine
- SOA Specifications

Biztalk case studies

2007-07-18T13:52:00.000-07:00

Here is a good link to BizTalk case studies. You can read some of the innovative ways in which some companies successfully made use of BizTalk server in their business needs.

.NET Code camp in San Diego

2007-06-29T15:25:00.000-07:00

I am attending SoCal .NET code camp, tomorrow, in San Diego. Brain Loesgen (a Principal Consultant with Neudesic) and Raman Koovelimadom (an independent consultant with lot of experience) are presenting some of my favorite topics:

- A look at Microsoft's ESB Guidance
- Building an Enterprise Service Bus (ESB) with WCF - Part 1
- Building an Enterprise Service Bus (ESB) with WCF - Part 2
- The Role of Microsoft BizTalk Server in a Service-Oriented Infrastructure

I will update you guys with my findings just after the camp.

Microsoft’s take on Enterprise Service bus(ESB)

2007-06-29T15:16:00.000-07:00

Like Service oriented architecture(SOA), Business process Management(BPM) ; Enterprise Service Bus(ESB) is another used, re-used and over-used phrase. Many companies defined this ESB in ways to support the new versions of their products.

Here are some of the contrasting definitions:

"A Web-services-capable infrastructure that supports intelligently directed communication and mediated relationships among loosely coupled and decoupled biz components." –Gartner Group

"The ESB label simply implies that a product is some type of integration middleware product that supports both MOM and Web services protocols." –Burton Group

"A standards-based integration backbone, combining messaging, Web services, transformation, and intelligent routing." –Sonic Software

"An enterprise platform that implements standardized interfaces for communication, connectivity, transformation, and security." –Fiorano Software

"To put it bluntly: If you have WebSphere MQ and other WebSphere brokers and integration servers, you have an ESB." –Bob Sutor, IBM

"The Enterprise Service Bus is a uniform service integration architecture of infrastructure services that provides consistent support to business services across a defined ecosystem. The ESB is implemented as a service oriented architecture using Web Service interfaces."
–CBDI

Microsoft was indifferent to the newly coined (not any more) phrase for some time, and kept working and improving it’s existing “servers and frameworks”. But recently it decided to take this bus before it misses it. (Joking) Per Microsoft, a combination of it’s BizTalk Server 2006 and .NET 3.0 framework( Windows Communication Framework to be more specific) provide a value which surpass the features provided by different vendor implementations of Enterprise Service Bus.

The following is a quote from the Microsoft ESB Guideline documentation:

"Microsoft ESB Guideline, contains architectural guidance, patterns and practices, and BizTalk Server and .NET components that enable Microsoft partners to build large and small-scale ESB solutions on Microsoft Application platform.

Microsoft ESB Guidance consists of a number of BizTalk projects including:

· ESB Core Engine
· ESB Core Services (dynamic transformation, dynamic routing, dynamic endpoint resolution)
· ESB Portal framework
· Exception Handling framework
· ESB Client application
· Namespace Resolution pipeline component
· JMS (Java Message Service) Interop pipeline component

Microsoft ESB Guidance also provides developers with a set of frequently encountered ESB use cases and sample scenarios that are maintained within the ESB Client application and can be executed with a dynamic set of parameters to demonstrate the functionality of the ESB Core Engine, the Core Services, the ESB Management Portal, and the pipeline components. Built using a modular architecture, many of the Microsoft ESB Guidance components can be deployed as stand-alone building blocks, providing ESB developers with the flexibility to customize and extend the standard reference implementation to match their specific SOI requirements."

Biztalk 2006 posters ...

2007-06-18T13:19:00.000-07:00

BizTalk Server team recently released very informative Biztalk posters.You can get lot of information about
- BizTalk Server 2006: Capabilities
- BizTalk Server 2006: Runtime Architecture
- BizTalk Server 2006: Legacy Modernization with Host Integration Server 2006

Check out BizTalk Posters.

Applying Microsoft technologies to vertical markets

2007-05-18T13:17:00.000-07:00

I believe that understanding the vertical market in which you are working; like the business forces in the market, the standards based consortiums that influence the vertical and the current (and future) techologies which can help solve a business problem; is critical for architects and designers. Such understanding helps the architects to visualize the standard business practices in the vertical market and design a solution that effectively translates business needs into technical requirements.

I really liked the new section of MSDN that includes the "Industry Centers". I suggest you to check out the vertical related business needs and the products of Microsoft that are useful in solving them.

You can make some interesting observations about
- vertical market
- Communication channels
- Standards based messaging
- Business process managment
- Data stores
- Application integration
- and usage of Microsoft technologies

BizTalk Vs Windows WorkFlow Foundation

2007-04-30T17:10:00.000-07:00

Many have asked me this question – “ Now that .NET 3.0 framework is out, does it mean that we are seeing the end of BizTalk server”.My answer is always the same, “ you are comparing apples to oranges. You are Comparing a framework to Server”.

In the current posting I would like to elaborate it.

Framework/Platform Vs Commercial product(Server)?

First of all, what is the difference between a framework and a server. I went to google to get a good definition for them. I found the following…

software framework:

“A software framework may include support programs, code libraries, a scripting language, or other software to help develop and glue together the different components of a software project. Various parts of the framework may be exposed through an API.” – Wikipedia

Windows Workflow Foundation:

Windows Workflow Foundation(WF) is a programming model, engine and tools for quickly building workflow enabled applications on Windows. It consists of a .NET Framework version 3.0 namespace, an in-process workflow engine, and designers for for Visual Studio 2005. WF is available for both client and server versions of Windows. WF includes support for both system, workflow, and human workflow across a wide range of scenarios including: workflow within line of business applications, user interface page-flow, document-centric workflow, human workflow, composite workflow for service oriented applications, business rule driven workflow and workflow for systems management.” – Microsoft MSDN.

BizTalk Server:

“A server, is an application program that accepts connections in order to service requests by sending back responses. Microsoft BizTalk Server is a server product by Microsoft which provides the following functions: Business Process Automation, Business Process Modeling, Business-to-business Communication, Enterprise Application Integration and Message broker[1].” – Wikipedia.
“ BizTalk is a business process management(BPM) server that enables companies to automate and optimize business processes This includes powerful, familiar tools to design, develop, deploy, and manage processes.” – Microsoft.

So .NET framework is very generic by nature. It supplies you with set of libraries and API which you use to implement a program in your favorite programming language. But Microsoft offers a wide range of servers which are offer a solution to a very specific problem. Also, a server built using a framework in the background. BizTalk server 2006 is built on .NET 2.0 framework. That’s why you cannot compare a framework with a commercial server. Remember, I also said that – a server is built on a (using a ) framework. So a server implements or extends the API that the framework exposed.
Now, you might say – “Fine, but I still see lot of overlapping features between these, what about that?”

Feature comparison:

Let’s compare some of the overlapping features of BizTalk server and Windows Workflow Foundation( which is now part of .NET 3.0 framework) Brian Losgen’s blog has a very good comparision of the features.I went further to highlight the some of the distinctions between these:

Hosting – “BizTalk server” , like the name suggests is a server to which clients connect using a specific adapters, on the other hand “Windows Workflow” needs to be hosted in a client like ASP.NET/Windows Forms/Windows Service/Console clients. WF programs have no independent existence.

Scalability – This non-functional requirement is satisfied by BizTalk inherently. BTS provides the concepts of host instances and clustering of Message boxes. But in WF it needs to be implemented in the hosting application by the developer.

Transactions – BTS has huge selling point by providing long term and short term transactions inherently. Such feature is absent in WF. It can be implemented in the WF host by doing a custom coding.

Cross-platform integration capabilities – BTS has lot of inbuilt adapters for different transports (HTTP , FTP, SMTP etc.) , different Line of Business applications (SAP, salesforce.com, PeopleSoft etc.) and to different accelerators different market segments ( HIPAA, HL7, SWIFT etc). BTS also provides extension points for custom adapters. The goal of WF is not really about adapters management. If a custom adapter were needed then it needs to be built outside of the WF context.

Management and administration – Again BTS excels in this. It has a very robust single point of administration. It’s administration console can perform things like – Application administration, adapter management, massage flow management and lot more. But WF does not have this feature. It relies on the hosts to manage these. For example, ASP.Net hosted workflows can exploit IIS management.

Extensibility - This is one feature in which WF excels in. If a custom activity or a shape needs to be implemented then, WF provides you with the API to do it. But, BTS is far more restrictive in this. BTS can only be extended in a very specific locations.

Transformation – BTS Mapper facilitates the developer to visually map nodes from one XML format to another. Internally it uses XSLT to transform the XML. It is a very useful feature in XML based applications. But, this needs to be implemented from ground up in the case of WF.

Types of workflows – WF is a framework for building different types of work flows like – system based, human/state based, rules based workflows. BTS is famous for it’s system based workflows. I tried to find a good reason for Microsoft for not really concentrating on human based workflows, and also for discouraging the use of them in BTS. Now , I know why, because, they are in the process of using the rich features of WF. This is another indication of usage of WF in the future releases of BTS. ( BTS vNext which is going to come after BTS 2006 R2).

Business Activity Monitoring – It is a very crucial feature in an enterprise context. It monitors the analyzes the key performance indicators of critical business processes.
This feature needs be implemented out side of WF context.

And there lot of these features exclusively built in a commercial product like BizTalk. I can see that, some of the features of BTS and WF overlap. But I suggest you to visualize them as complimentary. And also remember that the features of WF are not just limited to BTS, but will be used in Microsoft’s other commercial offerings like – MS Content Management Server, MS Commerce Server, MS Share Point Server etc. But the workflows built using WF deceiving look close to BTS Workflows, because some the BTS product team also worked on WF of .NET 3.0 framework.

The following rules of thumb can be used while deciding between WF and BTS. (Taken from MSDN article - "Architecting Enterprise Loan workflows and Orchestrations").

Use Windows Workflow Foundation When:

- An application itself will host workflows. Windows Workflow Foundation lets workflow be built into an application, allowing the workflow to be deployed and managed as a native part of the application. Conversely, because it's focused on integrating diverse applications instead of providing a general workflow framework, BizTalk Server always runs orchestrations within the BizTalk Server process that is external to the application.

-The business process being implemented requires human workflow. BizTalk Server addresses system workflow, and so it lacks Windows Workflow Foundation's support for things such as state-machine workflows and dynamic update. A scenario that requires both human workflow and more complex system-integration services could be addressed by using Windows Workflow Foundation and BizTalk Server together, however. For example, the Microsoft Office 2007 support for document-centric workflows, based on Windows SharePoint Services, might be used for the human aspects of the problem, while BizTalk Server handles the system integration aspects. The two can interoperate using the BizTalk Server Adapter for Office SharePoint.

-The workflow will execute on a client system. BizTalk Server is a server-focused product, and so it's less well-suited to run on desktop machines.

Use BizTalk Server When:

- Solving an EAI problem that requires communication with diverse applications on diverse platforms. Because of its focus on cross-platform integration, a large set of adapters is available for BizTalk Server that allows communication with a range of other software. Windows Workflow Foundation is focused solely on workflow, not EAI, and so it does not provide these things.

- B2B services are required. Windows Workflow Foundation does not address this area, while BizTalk Server provides tools for working with trading partners, accelerators for Mortgage Industry Standards Maintenance Organization (MISMO), Society for Worldwide Interbank Financial Telecommunication (SWIFT), and other industry standards.

- BPM services, such as Business Activity Monitoring (BAM), are required. While the Windows Workflow Foundation tracking infrastructure can be used to create these services, BizTalk Server provides important extras, such as tools that let information workers define their own BAM views.

- A complete management infrastructure and support for increased scalability are required. Unlike Windows Workflow Foundation, BizTalk Server includes a full set of tools for administering and scaling a production environment.

Finally, couple of words of wisdom from me - "use Windows workflow Foundation for building workflows within applications.Use BizTalk for building workflows across applications”.

References:

http://blogs.msdn.com/luke/archive/2006/12/13/community-content-for-wf-wcf-and-biztalk-server.aspx
http://geekswithblogs.net/bloesgen/archive/2005/10/09/56481.aspx
http://blogs.msdn.com/chunyu/archive/2005/12/13/503382.aspx
http://blogs.msdn.com/lamonth/archive/2005/09/14/466222.aspx
http://blogs.msdn.com/darrenj/archive/2005/09/15/467838.aspx
http://geekswithblogs.net/danielcarbajal/archive/2006/09/10/90825.aspx
http://blogs.msdn.com/mikewalker/archive/2007/04/24/biztalk-vs-wf-in-the-enterprise.aspx
http://biztalkland.blogspot.com/2007/03/biztalk-or-wf-vs-biztalk-and-wf.html
http://blogs.msdn.com/scottwoo/archive/2005/10/10/479331.aspx
http://www.sharepointblogs.com/heliosa/archive/2007/02/07/19152.aspx
http://msdn2.microsoft.com/en-us/architecture/bb330937.aspx

Web service Logging - Make or Buy decision - Part 3

2007-04-29T08:30:00.002-07:00

When you start thinking about logging then, the next question that pops up in head is this - should I make it or buy it?

For a moment let's talk about a problem which is much bigger than the current decision about logging of web services.

"Make or buy" is a much bigger problem in managment, they are not just software decisions. Hundreds of articles are written in the same subject. There concepts can be reused in many software development scenarios. Here are couple of articles I found in google by searching for less than 5 minutes. :)

http://www.uwnyc.org/technews/v5_n4_a2.html
http://www.developer.com/mgmt/article.php/1488331

My only suggestion is that - don't make any strategic decision based on a short term thought. Particularly with web services the decisions that you make are going to haunt you for a long time. Simply because of the following reasons:
- the web services based protocols are in evolution,
- the web services based frameworks are not 100% yet,
- vendors are still trying to grab a piece of this cheese by releasing intermediate version of their offerings
- Power of Service orientated architectures is not fully understood and implemented to the optimal level.

The above statements are not exactly related to logging of web services, but they are relevent in the bigger SOA move that many enterprises are making. And I feel that the web services play a bigger role in this SOA move.( Also, making web services succesfully does not mean that you are strictly conforming to SOA principles).

So , my suggestion to you is this - think of SOA for your enterprise and locate the role of these custom ( built or bought) web services frameworks in the bigger context. Also, instead of making depratment level decisions about the web services, have an enterprise level committe decide these moves. This will avoid the mayopic decisions about web services, instead it will open your creative eye for business process reuse.

Now, let's come back to the real topic of the blog posting. "Logging in web services". For those who are not really interested in my above spiritual and vague ramblings.... Here is a very simple test code that uses SOAP Extensions to log the messages received by a ASP.NET 2.0 ASMX web services in SQL data base. Also, you can download the code from my web site. The code like this will solve a very simple and short term problem of logging. But, I suggest you to think of a good web services managment frameworks which is also going to support your logging of web service.

Logging in web services context – Part 2

2007-04-27T11:49:00.000-07:00

In my previous blog posting, I discussed the relevence of logging in an enterprise. In this posting,I would like go into the technology perspective behind it. So let’s discuss about – How do we do logging in an application, in a sub system, in a department and in the enterprise? Do we affect the application in any way by doing logging? What are best practices in logging? Build or buy decisions that we need to take?

How do we do logging in an application, in a sub system, in a department and in the enterprise?

All right, we know that logging is good. But, let’s think about this scenario…

Consider an enterprise with multiple departments, and each with multiple applications. Just for fun, let’s picture it as below.

These is a good need to log each application in the enterprise, because each application is written with a business purpose. So, when each application logs into it's own proprietary data store then there will be “n” data stores for “n” applications. Imagine what would happen in a very large enterprise.. The production support team(s) need to maintain “n” more data stores in production. In order to get a single report these department managers need to run behind these “n” applications, collect the statistics of the key performance indicators, aggregate them and summarize them. This is a major problem.

After going through this, the smart architects of individual departments start thinking of a common logging framework. I deliberately chose the words “smart architects of individual departments”. I have seen logging frameworks developed for single department not sharing it with other deprtments (Don’t mean to be derogatory, it does happen in real life). Now it looks like this..

“Ideally”, it needs to look like this:

Can this “ideal” situation ever happen. Remember unless you are starting a brand new company and thought about this when you release the first application. It almost never happens. The enterprises are started even before IT ever existed, and all the new appliocations back then are now legacy systems which everyone is afraid of touching.

So in reality, we have some applications in production which cannot be changed, there are some packaged applications which have their own logging, some applications do have some funding and they can be changed in the upcoming versions; and finally, there are some brand new applications. This is the real scenario, and we have to accept it. Now, tell me about your logging framework again. … :)

So the answer to this problem my friend is this… think carefully and strategically.Try to exploit every opportunity to put the applications together. The problem of logging is much bigger than it appears. The right person to solve this problem is the “Enterprise architect” who has the visibility to the enterprise development, not the application architects nor the project-level managers.

Do we affect the application in any way by doing logging?

Yes of course.

It affects every application that it touches:

Let’s take a web service to illustrate the situation - “Customer web service” which provides a very critical and a comprehensive informtion about customer’s address, bank accounts, credit history, and driving history. While processing this request, “Customer web service” would call few systems. Say mainframe gives customer’s address and Finance application gives bank accounts and credit history, and driving history comes from some other system. Say the enterprise portal makes a call to this web service then, the call needs to be logged in all the these applications - Front-end portal, in Customer web service, in mainframe ,in financial package and in any other system/application. Unfortunately these applications are maintained by different departments. Now you see that, enterprise scale logging not a simple problem to solve.

Performance Vs logging:

Also, if you start logging every web method call, then there will be some negative consequences too. The first one that logging affects is the performance. Now, while serving the client web service needs to do two things – perform the needed action and also log the call in a central logging system. The central logging could be slow or too many routers away from the current web service hosting machine. It is even worse in the peak loads.

Oh god! Should we implement logging at all after all this. I still say yes. It is the necessary evil that we have to deal with no matter what. It is like a bad tasting pill that you need to take when you have fever.

Now that, we have to do logging.... Let’s figure out how.

Logging in web services context

2007-04-27T09:04:00.000-07:00

What is logging? What is logging in web services context? Do we really need it? What’s the benefit of it? Let’s discuss…

Logging to me is registering certain relevant events in a repository. Doing so we can revisit the repository, and retrieve some important performance of the key indicators. I would like to approach it from 10,000 ft. high.

We are living in a world which transitioned from single monolithic application management to enterprise-scale distributed interoperable systems. We also took out enterprise level processes out of secure locker and shared them with the suppliers and business partners. With the distributed, loosely coupled and interoperable systems; grew the complexity of maintenance of them. Technically, Web services are at the interfaces of these distributed systems working hard to take the messages and supply the responses. The value proposition from a business perspective, for a manager, is that - different departments in a large enterprise are communicating seamlessly and benefiting them financially and improving the time to reach the customer. With that said, it would be ideal to have a good monitoring system would help both technical folks and business managers.

How would logging help the technical person… just to give you an idea…

- by logging the exceptions in a sub system.
- By giving the speed of processing of each of the requests
- Can tell them whether a system is being used by clients. ( In other words, it will indicate whether system that was built by spending thousands of dollars is being used at all. Kind of relevant is not it?)

This list can grow much larger, as I start listing every good reason.

Similarly, the managers might know..

- Processes being integrated in the enterprise
- Performance of their processes , in terms of number of clients being served, the time it took to process each client request.
- Effectiveness of each project that they sponsored. Etc.

Now you got the point .. a good monitoring system will not hurt the enterprise, instead it will help the enterprise. And it is more relevant as the complexity of the systems grow and also as the number of systems being integrated grow. So, you do agree that it is necessary in a big corporate, right? … Right.

And, now you might say, how is this philosophical outburst relevant to logging web services. For those folks, I would like to say this – service orientation based web services are the interfaces to these complex sub systems and are constantly being criticized for failures in their service level agreements(SLA) conformance.

I don’t know whether it is just me, or you too, might have seen the poor web services being criticized for being too slow, failures that they cause and changes of the interfaces that the clients are not notified of…. It is tough to be the middle guy. So, it is very important to protect them (web service). Now, you might think that I have a point. I know, I am glad I convinced you.

Finally, how do you implement logging in web services… it is a topic for another blog discussion, and we will do it soon. – Chow.

Service Orientation based design of services

2007-04-22T08:18:00.000-07:00

I gave this presentation for one of my clients who were in the process of implementing enterprise scale integration broker architecture. The goal of this presentation is to go indepth about the usage of Service Orientation in desinging enterprise scale, contract driven, interoperable services. The content of the presentation is based on book named "Service Oriented Architecture:Concepts, Technology, and Design" - written by Thomas Erl. Download

A great introduction to .NET 3.0

2007-04-17T14:35:00.000-07:00

Folks, If you are getting ready for some serious .NET 3.0 stuff, you must start with the NETFX3 site that is been created. http://www.netfx3.com/
WF: http://wf.netfx3.com/
WCF: http://wcf.netfx3.com/
WPF: http://wpf.netfx3.com/
WCS: http://cardspace.netfx3.com/

.NET 2.0 to .NET 3.0

2007-04-14T07:53:00.000-07:00

If you are a developer like me learning and working feverishly on .NET 2.0, then such releases (.NET 3.0) in quick succession would definitely frustrate you. But the good news is that, .NET 3.0 is not going to put you back to zero. It is supposed to be an extension to .NET 2.0. All the stuff that you are currently working on should ideally work on .NET 3.0 framework with out any major re-haul. And, the formula is:

.NET 3.0 = .NET 2.0 + (WCF + WF + WPF + WCS)

WCF: Windows Communication Foundation (Formarly known as Indigo)
WF: Windows Workflow Foundation
WPF: Windows Presenation Foundation
WCS: Windows Card Space

It is not as bad as the move you made from VB6 to .NET. .NET 3.0 looks as below:

Finally ...

2007-04-14T07:45:00.001-07:00

I cannot believe that I finally have my own blog. After lot of slacking, I decided to start blogging. From few years I have been putting lot of my personal time and energy in gathering understanding enterprise technologies. I always wanted a full duplex communication channel which would broadcast and listen my noise at the same time. This blog is the outcome of my outburst. In this blog I am hoping share my views and listen to the opinions of my peers and mentors. The goal of this blog is to "learn, share and grow together".