Jdbc adapter's

The adapter supports integration of databases that are accessible through the JDBC application programming interface (API) with applications running on IBM® Business Process Manager or WebSphere® Enterprise Service Bus. The adapter provides outbound and inbound processing under the Java 2 Platform, Enterprise Edition (J2EE) Connector Architecture (JCA) and integrates with Service Component Architecture (SCA) components.

Outbound processing enables an application to access or modify data in a database. The adapter converts a request from the application to an outbound operation, which it runs to create, retrieve, update, or delete data in the database or to run a database program stored in the database. Processing these requests results in the creation, retrieval, update, or deletion of rows in the corresponding database tables. The adapter also enables you to run stored procedures or stored functions that are defined in the database, and to run user-defined SELECT, INSERT, UPDATE, and DELETE statements. You can use the adapter to integrate multiple applications with the same database.

Figure 1 presents an overview of the flow of outbound processing.

An application running in IBM Business Process Manager or WebSphere Enterprise Service Businvokes a service in an outbound module, which sends a request to the adapter to process one or more business objects. The adapter uses the JDBC API to connect to the database server, which accesses the tables and other objects in the database.

Figure 1. Processing outbound requests

Inbound processing enables an application to receive notification when objects in the database are changed. For example, an application can be notified when rows are created, updated, or deleted in selected database tables.

Figure 2 presents an overview of the flow of inbound processing.

A database application changes tables in the database. The change causes a trigger, or another automated mechanism, to update the event store with information about the change. Periodically, the adapter polls the event store, retrieves, and processes events, and then delivers them to the export of a module that is part of an application that runs in IBM Business Process Manager or WebSphere Enterprise Service Bus.

Figure 2. Processing inbound events

The adapter can process events in one of the following ways:

Standard event processing, using an event store that is populated by the database application

Custom event processing, using a user-defined database query

During standard event processing, when data is changed in the tables in the database, appropriate events are inserted into a database table called an event store, along with relevant information, such as key values. To capture the changed data, you can place triggers on the respective tables, or use other methods such as Oracle Change Data Capture, which is provided for Oracle databases. The adapter polls the event store and retrieves a batch of events. The events can be filtered by business object type and time stamp and connector ID. The adapter uses each event to construct a business graph or business object that contains the business objects changed by that event. The business object or business graph is then dispatched to the exports that are configured to receive the specific business object.

During custom event processing, the adapter runs a query that was specified by the user as a standard SQL statement, a stored procedure, or a stored function. Any of these actions returns a result set for data returned by the query. Each row of the result set corresponds to a row in the event store. The adapter constructs a business object for each event and delivers it to the exports (also called endpoints) that are configured for (or have subscribed to) the specific business object.

For both standard and custom event processing, you can specify how often the adapter polls for events and how many events it retrieves each polling period.

Outbound processing

When an application component needs to query the existence of a record in the database or retrieve or modify data in the database, the adapter acts as the connector between the application component and the database. The adapter provides a set of standard outbound operations, which process either after-image or delta style business objects. The adapter also supports both local and XA (distributed) transactions for outbound processing.

Inbound processing

The adapter supports inbound event management with event delivery. Events are processed from an event store that is populated either by the database application or from the result of custom queries that you provide. You control how often the adapter polls for events and how many records are delivered to the export at one time.

Business objects

A business object is a structure that consists of data, the action to be performed on the data, and additional instructions, if any, for processing the data. IBM WebSphere Adapter for JDBC uses business objects to represent tables and views in the database as well as the results of database queries, stored procedures, and stored functions. Business objects can also create a hierarchy of objects from your database and group unrelated tables. Your component communicates with the adapter using business objects.

Tables, views, and synonyms overview

Databases provide the common database objects that include tables, views, and synonyms.

Stored procedure overview

A stored procedure can be a business object that your module runs with the Execute operation. It can run in place of the standard SQL for an operation on any business object, or it can perform additional actions before or after performing an operation.

Stored procedure business object overview

You can create a stored procedure business object that corresponds to a stored procedure or stored function in the database. You can then use the Execute operation to run the stored procedure against the data in the database.

Stored procedures used in place of or in addition to operations

You can specify that the adapter use a stored procedure in the database in place of, before, or after the SQL statements that the adapter uses to perform an operation. Each business object can have a different set of stored procedures used with each operation.

Stored functions overview

Some databases support stored functions in addition to stored procedures. Stored functions are like stored procedures except that they always return a value. The adapter supports them in a similar manner.

Query business object overview

Query business objects run a user-defined SELECT statement against the database and return the matching records in business objects.

Batch SQL business object overview

A batch SQL business object runs one or more user-defined INSERT, UPDATE, and DELETE statements and returns the status of the statements.

The external service wizard

Use the external service wizard in IBM Integration Designer to discover objects in a database; to generate batch SQL, query, and wrapper business objects; and to generate business objects from selected database objects. The wizard also generates the module and the service artifacts that enable the adapter to run as a Service Component Architecture (SCA) component.

Log and Trace Analyzer

The adapter creates log and trace files that can be viewed with the Log and Trace Analyzer.

Business faults

The adapter supports business faults, which are exceptions that are anticipated and declared in the outbound service description, or import. Business faults occur at predictable points in a business process, and are caused by a business rule violation or a constraint violation.

Parent topic:Overview of IBM WebSphere Adapter for JDBC

Related tasks

Performing the migration

Creating the project

Setting deployment properties and generating the service

Setting deployment properties and generating the service

Related reference

Migration considerations

(C) Copyright IBM Corporation 2005, 2011. All Rights Reserved.
This information center is powered by Eclipse technology. (http://www.eclipse.org)

Performance Tuning

1) How you are integrating with MA systems (Sync or Async) ?

if it’s a Sync call, increase global transaction timeout , by default it will be 180 sec.

or

if it’s Async - you need to check your design that it’s really required to go with Async , as you said your transaction is waiting for long time. In ESB's you can set callout time.

At the moment, WPS making Sync calls to the backend system, and global transaction timeout in WPS server is 120 secs.

2) Have you guys did performance testing ?

No, there was no performance testing done on WPS server.

3) Connection pool size 10 is very low. Min 50 is recommended.

Yes, we are looking for the recommendation and can be considered. Can you please let us know , by increasing the connection pool size will there be any other issues like performance that may arise. As mentioned, WPS server is a stand-alone at the moment.

4) what is the connection pool type

Recommended to use non-XA data source if your data is not critical and you don't have auto roll back mechanism.

By default it was enabled in the WPS server, and we are logging each and every transaction in WPS server.

5) you need to increase maximum heap size to get rid of Out of Memory exceptions . 768 mb is recommended you can increase to 2 GB if you have enough RAM.

Currently we have minimum heap size as 4 GB and maximum heap size as 10 GB. And our current RAM is 42 GB and looking for some recommendation on this.

6) Ask WPS admin to increase thread pool size. It can also be the reason for hung threads/out of memory exception.

Web Container thread pool size is minimum 50 and maximum 50 and MQ thread pool size is minimum is 1 and maximum is 40. But we are looking for the recommendation here to increase the thread pool size.

Qualifiers

Interface Qualifiers

• Data Validation

 Determines whether the input data should be validated against its definition (ie. the XSD that describes the data).

• Join Activity Session

Determines whether or not the hosting container will join any propagated (client) activity session.

• Join Transaction

Determines whether or not the hosting container will join any propagated (client) transaction.

• Security Permission

Specifies a role, which is a semantic grouping of permissions that a given type of users must have to use an operation in an interface. The identity of the caller must have this role in order to be permitted to call the interface or operation. If no security permission is specified, then no permissions are checked and all callers are permitted to call the interface or operation.

• Store And Forward

Determines if messages should be stored when an exception occurs.

Store and forward qualifiers may be be set on Interfaces of Components, Imports or SCA Exports that can potentially be invoked asynchronously.

Implementation Qualifier

• Activity Session

Determines if the component's processing will be executed under an activity session.  An activity session is an alternative unit-of-work scope to the one provided by global transaction contexts. An activity session context can be longer lived than a global transaction context and can encapsulate global transactions.

• Security Identity

Specifies the identity under which the implementation executes at run time.

• Transaction

Determines the logical unit of work that the component's processing executes. For a logical unit of work, all of the data modifications made during a transaction are either committed together as a unit or rolled back as a unit.

Reference Qualifier

• Asynchronous Invocation

Determines if asynchronous invocations should occur as part of any client transaction. 

• Reliability 

Determines the quality of an asynchronous message delivery. In general, better performance typically means less reliable message delivery. With an Assured specification, the client application cannot tolerate the loss of a request or response message. With a Best effort specification, the client application can tolerate the possible loss of the request or response message.

• Request Expiration

The length of time after which an asynchronous request will be discarded if it has not been delivered, beginning from the time when the request is issued.

• Response Expiration

The length of time that the runtime environment must retain an asynchronous response or must provide a callback, beginning from the time when the request is issued.

• Suspend Activity Session

Prevents the propragation of the activity session context to a target component.  This applies when the target is invoked using the synchronous invocation.

• Suspend Transaction

Determines whether the invocation of a target service runs completely within the client's global transaction.  This qualifier only applies to synchronous invocations.

SOA

SOA (Service Oriented Architecture)

D 1: Used to design and develop a wide range of services which can be inter-operable.
D 2: It adopts component based programming model.
D 3: It is a set of principles and methodologies for designing and developing software in the for of inter-operable
D 4: Service-oriented architecture (SOA) is an approach used to create an architecture based upon the use of services. Services (such as Restful Web services) carry out some small function, such as producing data, validating a customer, or providing simple analytical services.

SOA is an architectural concept and web service is one of the technical approach to complete it,

Services

D 1: Repeatable business task and composed in to a business process.
D 2: Reusable ,defined with particular articrafts
D 3: Services are the building blocks and they are composed in to a business process.
D 4: Services are used to help get the right information to the right people at right time.
D 5: Service is nothing but a task or a method that is used to execute an activity,
D 6: Services can be reused and combined to deploy composite application to address to address new opportunities.

Service Characteristics 

• Well defined interfaces.
• Well defined quality of service capabilities.
• Well known endpoints: As a destination on the n/w that receives service requests.

Service Register

Locating and publishing web services.

Architectural Components

Figure 1 shows the basic components of a service-oriented architecture. The components of a service-oriented architecture include:

• Service providers. A service provider is a component or set of components that execute a business function in a stateless fashion, accepting predefined inputs and outputs.
• Service consumers. A service consumer is a set of components interested in using one or more of the services provided by service providers.
• Service repository. A service repository contains the descriptions of the services. Service providers register their services in this repository and service consumers access the repository to discover the services being provided.

Figure 1. Service-oriented architectural components

Principles of SOA

Modularity: 

• Component of a larger system that operates independently from the operation of the other components.

Encapsulation:

•  A service hides information or implementation details by encapsulating the information into a construct which presents an interface
•  Many services which were not initially planned under SOA, may get encapsulated or become a part of SOA.

Loose Coupling :

• Incompatible system technologies can be joined ,dis assembled just as easily in to their functional components.
• Services minimize dependencies on each other. 

Separation of Concerns/ Service Composability :

• Services break down a complex problem in a series of smaller pieces. Each of the smaller pieces address a specific concern.
• Services break big problems into little problems. 

Reuse:

• SOA is enabled through pluggable components with standard interfaces.
• Logic is divided into services with the intent of maximizing reuse.

Service Interoperability :

• Services should use standards that allow diverse subscribers to use the service. This is considered so obvious these days that it is often dropped as a principle. 
• 

Service Discoverability :

• Services can be discovered (usually in a service registry).
• 

Service Autonomy :

• Services should have control over the logic they encapsulate. 
• 

Service Abstraction :

• Services hide the logic they encapsulate from the outside world. 

Challenges of SOA

• Managing service metadata(Information about data,Data About Data)
• Lack of testing in SOA space.
• Providing appropriate security
• Interoperability
• Early adoption and evolution of supporting technology 
• Organization change in necessary since SOA crosses system boundaries 
• The architecture is enterprise in scope encompassing dispersed and heterogeneous systems 
• The infrastructure is distributed requiring high availability and scalability 
• The project life cycle methodology requires changes due to complex system dependencies, SOA specific design patterns, and the change impact to the infrastructure and users 
• Program management is often complex due the project scope, interdependencies and new technology risks 
• Quality Assurance is difficult since services are distributed, have many interfaces, require new testing environments, and message based testing tools 
• New competencies must be developed spanning project management, analysis and design, development and operations. 

Challenges

While implementing a service-oriented architecture, a company faces up to eight key challenges. These challenges align to the steps in a typical project deployment plan:

1. Service identification. What is a service? What is the business functionality to be provided by a given service? What is the optimal granularity of the service?
2. Service location. Where should a service be located within the enterprise?
3. Service domain definition. How should services be grouped together into logical domains?
4. Service packaging. How is existing functionality within legacy mainframe systems to be re-engineered or wrapped into reusable services?
5. Service orchestration. How are composite services to be orchestrated?
6. Service routing. How are requests from service consumers to be routed to the appropriate service and/or service domain?
7. Service governance. How will the enterprise exercise governance processes to administer and maintain services?
8. Service messaging standards adoption. How will the enterprise adopt a given standard consistently?

https://msdn.microsoft.com/en-us/library/aa480029.aspx

Interview Questions 2

What is difference between the parallel flow and generalized flow activities in BPEL process?

Parallel Flow

• A parallel process (or flow) is a collection of other process activities that are all nested within a parallel activity.
• The nested activities run sequentially in an order that is dictated by links and transition conditions (when no links are present, all activities will be executed concurrently).
• Parallel activities are very versatile, and can add a depth to a long-running process.

Generalized Flow

• A generalized flow activity is very similar to a parallel activity in that you can nest other process activities within it, and then control the execution order of those activities through links.
• The generalized flow activity differs in its ability to use conditional links to loop back to previous activities in the sequence.

How do you assign people to a process (people assignment)? What are the pre-defined roles for human tasks?

When you define a human task you must stipulate which people are eligible to view, initiate, perform and administer the task. This step is known as people assignment. You can assign work to a named individual, to a member of a certain group. There are pre-defined roles for human tasks which you can assign people or groups to. Members of a role share the same level of authority. The roles are:

Administrators - have the authority to perform high level duties like suspend, terminate, restart, force-retry, and force-complete.

Potential creators - can create an instance of the human task, but cannot start it.

Potential starters - have the authority to initiate an existing instance. The starter role is subtly different from that of creator, and although a creator can create a new instance, only a starter can start it.

Potential owners - can claim, work on and complete tasks.

Editors - can work with the content of a task, but cannot claim or complete it. For example, an editor can receive the work item to review a document and add comments, but an editor is not able to finish the task.

Readers - are allowed to view tasks, but cannot work on them. This role can be used in situations where someone needs to monitor a task without taking any action in it.

When do you decide to create a new version of BPEL process?

Here are some possible examples of when you would create a version of a BPEL process:

• In the likelihood that your BPEL process will need to be modified over time, but its callers will not. In such a case, you will want the existing callers to be able to seamlessly pickup the newest version of the process the moment it becomes effective.
• You have a solution where multiple versions of the same BPEL process must coexist. Although the solution as a whole cannot be uninstalled and reinstalled, you will need to be able to deploy new versions of the process in such a way as to ensure that new instances use the latest version and, wherever possible, existing instances also migrate to the latest version.

How do you handle faults or exceptions in BPEL process?

A fault is any exceptional condition that can change the normal processing of a BPEL process and, if not handled, can lead to undesirable conditions or results. A well-designed process is one that anticipates possible faults with fault handlers that are designed to lead to predictable outcomes.

What is Correlation or Correlation Sets? What is purpose of them?

• Correlations are used in runtime environments where there are multiple instances of the same process running.
• A correlation set for a BPEL process consists of a name and one or more properties. They are used to distinguish the instances of same process in runtime.
• Correlation sets allow two partners to initialize a BPEL process transaction, temporarily suspend activity, and then recognize each other again when that transaction resumes.

What are the transactional behavior options for the activities inside a BPEL process?

Commit before: preceding transaction commits before this activity starts

Commit after: transaction commits immediately after this activity is completed

Participates: activity runs in existing transaction

Requires Own: activity is isolated in its own transaction.

Usage of BSM vs BPEL process?

• In cases where the state machines are--and will--remain simple, it may be better to use a BPEL process. For example, a state machine without any loops (returning to an earlier state), would probably make more sense being developed as a process.
• However, if there is a lot of looping, the state machine provides a much more natural way to develop, debug, and monitor.
• If you do have performance constraints, then do not prefer BSM as it is internally will get converted into BPEL process by the Process Server runtime.

Compare the Rule sets and Decision Tables?

Rule sets model the typical if-then kind of business rules, if you need to model rules on values from multiple input combinations, then decision tables are preferred.

http://simplicable.com/new/choreography
http://simplicable.com/new/process-orchestration

How do Levels Works?

A log request of level p in a logger with level q is enabled if p >= q. This rule is at the heart of log4j. It assumes that levels are ordered. For the standard levels, we have ALL < DEBUG < INFO < WARN < ERROR < FATAL < OFF.

The Following example shows how we can filter all our DEBUG and INFO messages. This program uses of logger method setLevel(Level.X) to set a desired logging level:

This example would print all the messages except Debug and Info:

import org.apache.log4j.*;

public class LogClass {
   private static org.apache.log4j.Logger log = Logger.getLogger(LogClass.class);
   
   public static void main(String[] args) {
      log.setLevel(Level.WARN);

      log.trace("Trace Message!");
      log.debug("Debug Message!");
      log.info("Info Message!");
      log.warn("Warn Message!");
      log.error("Error Message!");
      log.fatal("Fatal Message!");
   }
}

When you compile and run the LogClass program, it would generate the following result −

Warn Message!
Error Message!
Fatal Message!

Server Hang

Change 1
[2/20/17 7:53:19:800 CST] 00000262 PartnerLogDat E   WTRN0000E: An internal error occurred in method logData in class com.ibm.ws.Transaction.JTA.PartnerLogData; the exception stack trace follows: com.ibm.ws.recoverylog.spi.InternalLogException
Line 114616: [2/20/17 7:53:19:803 CST] 000002e0 PartnerLogDat E   WTRN0000E: An internal error occurred in method logData in class com.ibm.ws.Transaction.JTA.PartnerLogData; the exception stack trace follows: com.ibm.ws.recoverylog.spi.InternalLogException
Line 114641: [2/20/17 7:53:19:807 CST] 000002e8 PartnerLogDat E   WTRN0000E: An internal error occurred in method logData in class com.ibm.ws.Transaction.JTA.PartnerLogData; the exception stack trace follows: com.ibm.ws.recoverylog.spi.InternalLogException
This may cause because our transaction/partner log is getting full. For temporary fix we can increase the size of partner log size. Please find below link for reference.

http://www-01.ibm.com/support/docview.wss?uid=swg1PM11632
https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014787263
Change 2
We have around 40 ActivationSpec and each ActivationSpec has maximum server session 2. So ideally WMQJCAResourceAdapter thread pool max size should be 80(40*2).But in our case it is 75.

We should do this change when we migrated few new modules.
We have to do change adaptor thread pool settings to 90 .Lets do this change next time before doing JVM restart.


http://www-01.ibm.com/support/docview.wss?uid=swg21383363
https://www.ibm.com/developerworks/community/blogs/aimsupport/entry/why_are_messages_piling_up_in_process_server_queues_and_not_getting_processed?lang=en 


http://www-01.ibm.com/support/docview.wss?uid=swg21115785

Interview Questions

What are the differences between long-running process and micro-flows? when do you use each of them?

We use long running process only in the following conditions:

If your process requires more than one transaction.If your process will need to stop at any point and wait for external input, either in the form of an event or a human task.If you have elements in your process that you would like to run in parallel.

We use short running process in the following conditions:

A Microflow is a process that is contained within a single transaction.Because it runs automatically from start to finish, it is ideal for situations where the user is expecting an immediate response.Example of a Microflow would be a database query.Microflows cannot be used for processes involving human tasks.If you have a short series of steps to model and want them executed very quickly in the runtime environment, use a Microflows.

Performance: short running processes offer great performance; so, where ever feasible one should use short running processes.

What is Human Task?

A human task is a unit of work that involves a person. Examples would be an review process, in which a manager must provide final approval, or when a follow-up telephone call with a client is required.

The definition of a human task includes the following information:

who can perform the taskwhat needs to be donewhat happens when the task takes too longhow the task will be done

What are different implementation mechanisms or subcategories of Human Tasks?

Stand-alone: A stand-alone task exists independently of a process, and implements human interaction as a service that can be wired to any other component.

There are two instances in particular when you should model your human task as a stand-alone task:

The task provides just another service.You intend to replace the stand-alone task at a later stage and do not want to change the component to which it is wired.

Inline: An inline human task is a piece of a larger BPEL process that must be performed by a person.

You should model your process with an inline task when:

You need information from the process logic to run the human task. Although information from the process can also be modeled into the input for a human task, the main reason to use an inline human task is because they have direct access to the process context without the need to explicitly model the required information into the input message.

You want to define authorization rights on specific activities.

What are different types of Human Tasks?

To-do task - a service schedules a piece of work for a person to perform.

Invocation task - a person uses a service.

Collaboration task - one person assigns work to another person.

Administration task - a person is granted administrative powers over an activity or process.

How do you handle faults or exceptions in BPEL process?

Here are some of your options for dealing with faults that are available in the BPEL process editor:

Use a terminate activity to stop the execution of a process or an activity so someone can step in and make necessary repairs.Use a reply activity with a fault name associated with it so it will respond with a fault.Use a throw activity to signal an internal fault.Use a fault handler to catch a fault and attempt to deal with it.Use compensation to roll back or undo a process or an activity that has failed after committal.

What is Compensation?

Compensation is the means by which operations in a process that have successfully completed can be undone to return the system to a consistent state if an error occurs.

You can compensate a BPEL process in two ways:

Save the properties of the individual parts of a process so that they can be restored if the process cannot be committed and must be rolled back (compensation pairs).Use a compensation handler to return a failed process to a balanced state after a fault is thrown when the parent activity has already been committed.

What are Escalations?

If a task is overdue it needs to be escalated. Use escalation properties to define when a task must be completed and the actions to take if deadlines are missed. There are three possible states for which an escalation can be configured:

Ready- When a human task is in a ready state, it is waiting to be claimed. You can configure an escalation to trigger if it sits unclaimed for too long.

Claimed- If a staff member has claimed a task, but takes longer than the specified period of time to complete it, an escalation is triggered and another staff member is notified, for example, the manager of the claimant.

Subtask started- A subtask is an additional unit of work that is split out from a parent task. If the subtask fails to complete within a specified period of time, the parent task is escalated and indicates that it is still waiting on the subtask.

How do you enter the input data required by the process?

We can enter the input data required by the process by using the Business Process Choreography Explorer. It is access using Web Browser.

What is rule group? How do you implement rule groups?

A business rule is a condition that must be satisfied when a business activity is being performed. A rule can enforce a business policy, make a decision, or infer new data from existing data.