Co-authored by BPM.com's Nathaniel Palmer, and with a forward by Dr. Bruce Silver, the BPMN 2.0 Handbook offers both the business and technical perspectives written by the standard's authors, leading implementors, and most respected experts; The 47-page excerpt contains the complete Forward, Introduction, BPMN Glossary, and Making a BPMN 2.0 Model Executable; authored by Nathaniel Palmer and Lloyd Dugan. Free to registered BPM.com members.
Click Here to Download
Articles
Improving Business Operations With Real-Time Information: How to Successfully Implement a BAM Solution
Business Activity Monitoring (BAM) provides real-time information to business managers and personnel for more effective, efficient management of operations. Information is visually presented through one or more operational dashboards and distributed in alerts via email, Short Message Service (SMS), fax, pager, and the like.
Distinct from other analytical systems, BAM solutions also provide built-in response capabilities that let users act directly from the dashboard to address a problem or opportunity without having to switch applications or manually respond. For anyone managing a fast-paced business, this type of system is extremely valuable, and the number of companies implementing BAM solutions has grown significantly.
This article describes how to successfully design and build a BAM solution, outlining the phases, challenges, and success factors. Let’s begin with the three types of scenarios where BAM solutions are deployed:
• Monitoring business processes: Companies implementing Business Process Management (BPM) solutions for automating processes or managing human workflow often use BAM systems to monitor their processes for exceptions, analyze aggregate throughput, find resource bottlenecks, track Key Performance Indicators (KPIs), and alert the appropriate personnel when KPI thresholds are exceeded. For example, a company processing loan applications can track how many applications are in-process or completed, calculate the average applicant credit rating or loan interest rate, and raise an alert if too many high-risk loans are approved within the last hour.• Monitoring transaction systems: For processes that aren’t automated or captured in a BPM system, companies use BAM systems to monitor the individual transaction systems that operate as part of those end-to-end processes. By monitoring all the transaction systems in a business process, BAM can provide end-to-end process visibility. For example, the order-to-cash process interacts with applications, including Customer Relationship Management (CRM), billing, and accounts receivable. By correlating customer numbers with order and invoice numbers across those applications, it’s possible to track the entire order-to-cash process in real-time and detect exceptions such as unmatched invoices.
• Business optimization: Business optimization scenarios are specific to different types of operations, but warehouse management is a good example. Warehouse managers use BAM systems to optimize delivery loads for their trucking fleet, even when shipments for delivery arrive in various volumes at different times throughout the day. The BAM system captures events from multiple sources, including new orders from the point-of-sale system and stockout alerts from the inventory application. BAM analyzes incoming events and either provides the appropriate drivers with new loading instructions or automatically optimizes the shipping system with a new delivery schedule. The common thread in all these scenarios is that businesses are using real-time information to better allocate resources, improve customer service, and reduce costs. Without real-time information, business managers and operations personnel must rely on general guidelines to make decisions, and that produces less-than-optimal results. BAM systems provide businesses a chance to establish a competitive advantage by optimizing their operations. Challenges Businesses have long sought real-time information, so why are BAM systems only now becoming more common? Historically, real-time information systems were limited to specific, high-end scenarios within traditional IT-heavy industries. These solutions were expensive and difficult to build because each was designed for a particular environment and often relied on customized message and network protocols. The growth of standard, reliable, and high-performance, message-based IT systems; less expensive hardware and memory resources; and XML standards for exchanging data made it possible to build real-time information systems for the broader IT market. Even with a modern, message-based infrastructure, companies can still face challenges when implementing a BAM solution. Usually, companies are challenged when attempting to build a BAM solution with their existing Business Intelligence (BI) or data warehouse systems. BI and data warehouse systems aren’t designed for real-time analytics; they use analytic processes that can take hours or days to refresh with new data. Attempting to force those technologies to work as a real-time, event-driven solution will rapidly escalate the implementation and maintenance costs. Expensive hardware and manual tuning might improve performance to some degree, but at significant cost. A better approach is using BAM and BI together; combining these provides a solution with both real-time operational data and deep, historical analytics. Now that we’ve set the context for a BAM solution, we’ll describe our recommended approaches to implementation. These apply whether you want to build your own solution or use a packaged offering with out-of-the-box BAM functionality. We’ll start with defining a vision. Define a Vision Begin your project by defining a vision of what real-time information could be used to improve your business operations. With this important step, you can specify how the real-time information, alerts, and response capabilities of your BAM solution will improve the business. Examples of how the BAM solution could be used include tracking the: • Number and type of open service requests for better allocating service personnel • Number and value amount of product orders per hour to measure effectiveness of sale promotions • Average time to complete a financial transaction to avoid penalty fees. If your organization has predefined KPIs or metrics that define Service-Level Agreements (SLAs), that data should be captured, too. Begin to visualize your dashboard, and spend one or two team sessions drawing it on a whiteboard and then on paper. You should expect that the dashboard will change as users become more familiar with the information it contains and discover new ways to use and organize it. Think about how different kinds of analytics, such as sums, averages, or custom-calculated fields can be visually represented through displays, bar charts, pie charts, and status indicators such as green-yellow-red stoplights and the like. Users are most comfortable with visual charts as a first-level display, but then want the capability to drill down into that information for detail. So the top level often contains primarily summary and graphical information, but from there, users should be able to access the granular detail of the data. The next step is to consider the possible actions users may want to initiate from the dashboard. Dashboards with built-in capabilities to initiate actions let users quickly respond without having to switch applications or manually respond. In addition, built-in actions let system designers specify a finite set of possible responses, which deters users from creating their own that might be inappropriate or inefficient. Furthermore, the BAM system can track the given responses and use that information to adapt its behavior, including the possibility of automatically taking action without requiring user intervention. Finally, consider that sophisticated BAM systems will let users customize or define dashboards themselves in an easy-to-use design environment that requires no programming. With the initial BAM system rollout, however, users will find it helpful to have some pre-existing dashboards so they have an idea of how the dashboards are constructed. Roles, Responsibilities and Success Factors A successful BAM implementation requires these resources:
• Architect: The architect designs the data model within the BAM system and identifies the data source systems, proper connection protocols, messaging patterns, and the like. In some organizations, this role may be split into a data architect and system architect.
• Developer: Often, a developer will be required to develop the datastreams from existing applications or IT systems. These data sources will almost always include internal systems, and will often include external systems from partners outside the firewall. Developers also may need to expose services from specific applications for the BAM system to invoke as part of a response action.
• Analyst: The analyst is responsible for defining the user dashboard and alert requirements. The analyst also is responsible for building the dashboards for the initial BAM system rollout. (Building BAM dashboards shouldn’t require coding or use of a development tool.)
• Users: Obtaining user input and feedback on prototypes is critical. Ensuring that users like the dashboard and can see the information they need minimizes the risk of low user adoption. Major design factors required for a successful implementation include.
• Performance: Performance is an extremely important factor in regard to delivering real-time information. Though there may be situations where “near real-time” information is acceptable, your BAM system should be capable of delivering information in actual real-time.
• Heterogeneous data access: A BAM system needs to capture data from multiple systems in parallel. A good BAM system can connect to multiple data sources quickly and easily.
• Usability: BAM users are business users, so the BAM system needs to be appealing and easy to use. Users should be able to easily see and consume real-time information, then act on that data by using the response capabilities built directly into the dashboard. Define the Data Model Once necessary information for the dashboards and alerts is defined, the architect designs the data model. It’s important to separate the data model from the dashboard itself, so the dashboard can be customized and changed without changing the data model. Building new dashboards should be easy, but changing the data model isn’t! The recommended data model for a BAM system is a star-schema, the same technique used for data warehouse systems. Using star-schema fact and dimension tables provides the best design for fast queries, which equals the best performance. As a real-time analytics system, BAM needs the best possible query performance. Set Up Real-Time Datastreams The next step is setting up real-time datastreams to propagate data from the source systems to the BAM server (see Figure 1). The ideal source for capturing real-time data is a message-based system such as Java Messaging Service (JMS) or MQSeries. These provide a well-defined message format that can reliably and securely transport data from applications to the BAM server. Part of the attraction of BAM systems is the prevalence of messaging throughout IT environments. Where messaging systems already exist, a BAM server can easily subscribe to the appropriate message queue and capture real-time information without adding processing overhead to the application or IT system. In many scenarios, a pre-existing message stream with all the necessary information won’t exist, so you’ll need to do some development to get data into the BAM system. There are two major options (see Figure 2): • Push data from the data source to the BAM server. This usually requires writing code in an application to call out to a message system such as JMS or even directly to the BAM server via a Web Service or native Application Program Interface (API). This option tends to be superior because it’s event-driven and no processing is required until there’s an event to propagate to the BAM server. • Automatically poll the data source at regular time intervals and pull back new data to the BAM server. This option isn’t event-driven and therefore isn’t the ideal BAM datastream. Polling for data creates inherent delays and increases the processing load on the source system. The polling option is common when data isn’t available directly from the source system, but instead is batched and stored into a file the BAM server can access. BAM systems will need to support both these scenarios. For data stored in a database table, use database change-notification capabilities to generate an event message for any table update. Some databases can provide this functionality without using table triggers with technology referred to as change data capture, continuous queries, or streams. Performance and reliability are key success factors for a BAM implementation. A large in-memory data cache for faster processing and analytics is at the core of a BAM system’s superior performance. The data cache is backed by a database persistence store that prevents data loss in case of system crash. To ensure that data is never lost, message or data input should never be acknowledged as received until that data has been persisted to the database. To improve performance in high-volume datastreams, those updates to the database should be batched. Building Dashboards and Alerts Dashboards and alerts are the ultimate information-delivery vehicles of a BAM system, so it’s important to make them useful and user-friendly. Web-based, thin-client dashboards work well because most users are comfortable with Internet clients. In addition, Web-based dashboards are easy to roll out and upgrade. The best solutions will use dynamic technology such as Asynchronous JavaScript and XML (AJAX) to update the display in real-time without requiring manual or automatic page refreshes. AJAX is a new Web programming technique that combines several technologies to make Web applications more interactive by exchanging data with the server without requiring the page to reload. Users who see their page changing dynamically will immediately understand the real-time nature of the information and how it can be leveraged. Users also need the ability to customize and define their own dashboards. Traditionally, users have rejected attempts to develop dashboards if it required writing code or installing new development tools. Implementing Web-based designs that let users easily set up data visualizations and lay out their own dashboards greatly increase the system’s appeal. Users also should be able to easily define their own alerts based on any metrics or data elements on the dashboard. An event-driven model for propagating data, this time from the server to the dashboard, is necessary to support real-time performance requirements in a high-transaction environment. Only events that affect changes to the dashboard display should be sent across the network from the server to the client. This requires the dashboard to maintain an open connection with the BAM server to receive these updates. Without an event-driven model or an open connection between the BAM server and dashboard, the dashboard is forced into a polling mode with slower performance and increased server load. Summary Businesses are improving their competitive advantage through more effective, efficient operations by leveraging real-time information. BAM systems are increasingly easy to build and deploy by leveraging access to real-time data in their applications and IT systems through standards-based integration and messaging infrastructure. Implementing those real-time information solutions requires a team that contains the right roles and responsibilities, focused on the key success factors of performance, heterogeneous data access, and usability. The key steps are to define a vision, establish the data model, build real-time datastreams, and roll out operational dashboards to users so they can customize their information display and define their own alerts. The result is an empowered organization that has access to reliable, real-time information for making decisions that optimize operations.
|
|
BPMN 2.0 Handbook
Most Downloaded
2009 BPM State of the Market Report (3405)- BPMN 2.0 Handbook (2915)
- Process Mapping 101 A Guide to Getting Started (1305)
- What is the difference between Workflow Engines and BPM Suites? (1278)
- Getting Started with Business Process Management (1243)
- WHITE PAPER: 180 Day Transfomation Plan (1157)
- Getting Started With BPM (1114)