Monitoring OpenJPA's Caches on the WebSphere Liberty Profile

OpenJPA provides several types of caches that can be used to improve the performance of many applications. Several of OpenJPA's caches collect statistics such as number of entries, hit count, and object type. These attributes can be helpful for monitoring the effectiveness of the cache.  Analysis of these statistics can help you to make decisions about what to cache, how many objects to cache, or even whether to enable caching at all.  An effective caching scheme can produce significant performance benefits, while an ineffective scheme may be a drag on performance and almost certainly increase the memory requirements of the application.  A little tuning can normally go a long way.

Initally, accessing OpenJPA's cache statistics required adding custom Java code to your application.  You were required to call OpenJPA APIs to navigate through the configuration, get access to the caches, and then access their statistics.  As of OPENJPA-1739 (OpenJPA 2.1), these caches are now JMX instrumented.  This means that a application such as JConsole can be be used to monitor cache information out-of-band.  This type of monitoring can be accomplished without changing the application code and with only minor changes to the configuration.  For example, by adding set of properties to your persistence.xml, you can enable instrumentation of the data cache, query result cache, and SQL caches:

<property name="openjpa.Instrumentation
    value="jmx(Instrument='DataCache,QueryCache,QuerySQLCache')"/>
<property name="openjpa.DataCache"
    value="true(EnableStatistics=true)"/>
<property name="openjpa.QueryCache"
    value="true(EnableStatistics=true)"/>
<property name="openjpa.jdbc.QuerySQLCache"
    value="true(EnableStatistics=true)"/>

Using JConsole (provided with some 1.6 and later JDKs), you can attach to your Java program and use the MBeans provided by OpenJPA to view statistics for its caches.  You can also invoke methods on the MBeans to get individual statistics about certain queries or cached object types.  In addition to direct monitoring of the MBeans, the OpenJPA project provides a data cache plugin for JConsole.  The data cache plugin provides a nice interface for monitoring the L2 data cache.

As long as the environment supports JMX, applications running Java SE and Java EE environments can be monitored. The OpenJPA documentation explains how to enable the JMX server in a Java SE environment. This blog contains some information for using JConsole with WebSphere Application Server full profile.

JPA applications deployed on the WebSphere Application Server Liberty Profile version 8.5 and the Liberty V.8.5.Next Beta can also be monitored. Configuration is a piece of cake. You simply need to add the configuration above to your persistence.xml and enable the localConnector-1.0 feature in your Liberty Profile server.xml:

<server description="Liberty Server">
    <featureManager>
    <feature>jsp-2.2</feature>
    <feature>jpa-2.0</feature>
    <feature>localConnector-1.0</feature>
    </featureManager>
    ... other configuration ...
<server/>

Start the Liberty application server, and begin using your application. After performing a few operations that use JPA, start up JConsole with the OpenJPA Data Cache plugin and connect to the server. You can find the Liberty Server in the Local process list by searching for for ws-launch.jar followed by the name of your server.



JConsole will connect to the local process and the data cache plugin will discover the PUs available on the server. A tab will be created for each PU it discovers. This screen shows the DataCache statistics for the PollPU persistence unit.



You can also select the MBeans tab and browse through cache statistics of all the caches for the various persistence units in the server. The MBeans also allow you to reset cache statistics and query statistics for individual objects or queries on the fly.



-Jeremy

EJB Lite feature supports JPA in Liberty Beta

The WebSphere V8.5.Next Liberty Profile Alpha gave us a first look at a new EJB Lite (ejblite-3.1) feature for the WebSphere Liberty profile.  If you use EJB session beans to drive your persistence layer you may have been disappointed since the EJB Lite capability in the Alpha didn't include support for JEE-style use of JPA with EJBs.  Attempts at injecting or lookup of JPA entity managers and/or factories within an EJB resulted in an exception.  The use of JPA with Java SE style bootstrapping (Persistence.createEntityManagerFactory()) may have worked if you happened to be using resource local transactions and connection properties (rather than server defined data sources).  This was not an ideal or typical configuration in the application server environment.

But, enough about the Alpha!  I'm happy to report that the recent WebSphere V8.5.Next Liberty Profile Beta includes a major bump in EJB Lite functionality, including integration with JPA.  This integration includes all you might expect, container managed persistence, enlistment in JTA transactions, and support for extended persistence contexts.  The WebSphere Development Tools for Eclipse Beta 9 on top of Eclipse WTP makes it a piece of cake to create new or modify existing EJB and JPA applications.  To enable EJB and JPA in the Liberty Beta just add this feature set to your server.xml (include any other features you may require):

  <featuremanager>
    <feature>ejblite-3.1</feature>
    <feature>jpa-2.0</feature>
  </featuremanager>

Check it out!

-Jeremy

OpenBooks sample works with new WebSphere Liberty Profile!

There is an exciting new feature in WebSphere Application Server v8.5 called the Liberty Profile.  The Liberty Profile provides a new, sleek, highly-efficient application server environment both for development and production use.  You can find out more details about Liberty on WASdev Home.

For those of you that joined me at Impact 2012 last week, I mentioned in my JPA presentation that I had attempted to deploy OpenBooks to a Liberty Profile server on the plane to Vegas.  With just a couple of modifications to my configuration, I was able to get OpenBooks up and running in no time.  But, I needed to update the OpenBooks instructions before going "public" with the information.

I have now completed the updates to the OpenBooks sample and associated instructions.  In addition to making the necessary configuration updates to run the OpenBooks application on Liberty, I also updated a few other build, install, and execution issues.  You can now build and execute the OpenBooks sample in JSE mode and two JEE environments -- WebSphere Application Server, and the new Liberty Profile.  Enjoy!

Lightweight JPA testing in a JEE environment using the WebSphere Embeddable EJB Container

Recognizing the importance of accelerated development and simplified testing, Enterprise JavaBeans took a page out of the JPA book by including JSE-friendly embeddable container within the EJB 3.1 specification. While the embeddable container doesn't support all the bells and whistles of a full blown container, it gives you plenty of features that can drastically speed up development and more importantly, provide a test environment that more closely resembles a JEE environment - in both configuration and behavior.

The EJB 3.1 specification outlines these features for the embeddable container:

• Synchronous calling of local stateless, stateful, and singleton beans (singletons are new in EJB 3.1)
• Declarative and programmatic security
• Interceptors
• Use of annotations or XML deployment descriptors
• Support for JPA 2.0

In addition to these features, the embeddable EJB container provided with WebSphere Application Server V8 provides the ability to easily configure JDBC data sources. This is especially important when using JPA in container managed mode. WebSphere's embeddable container even supports the use of JTA component data sources within your persistence.xml. This allows you to use the same persistence.xml with the embeddable container as you use in your enterprise application.

With the exception of an optional (but recommended) IDE such as Eclipse or RAD 8.0.3, WebSphere Application Server V8 contains everything you need to get started - the Java SDK (depending on platform), the embeddable container, JPA client package, and the Derby database. When you install the beta be sure to install Stand-alone thin clients, resource adapters and embeddable containers.



For the purposes of an example, let's say we have a simple EJB module that is used to manage personal contacts. The ContactModule contains a simple annotated session bean named ContactBean which is primarly a transactional facade over JPA operations. The operations provided by the EJB allow you to find a contact by name, save a contact, and clear all contacts. The JPA operations operate on a persistent entity, Contact. The ContactBean is a session bean that implements a local interface ContactLocal. While EJB 3.1 will now provide access to an bean without a Local interface, using interfaces is still a good practice. Interfaces provide a layer of abstraction from the base bean. They allow the bean to implement multiple specialized interfaces or allow the client to call a different EJB which implements the interface without modifying Java code. Here are our EJB and JPA classes and persistence.xml:

ejbs/ContactBean.java

package ejbs;

import static javax.ejb.TransactionAttributeType.REQUIRED;
import static javax.ejb.TransactionAttributeType.SUPPORTS;

import java.util.List;

import javax.annotation.Resource;
import javax.annotation.Resources;
import javax.ejb.Stateless;
import javax.ejb.TransactionAttribute;
import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;
import javax.persistence.Query;
import javax.persistence.TypedQuery;
import javax.sql.DataSource;

import entities.Contact;

// Data source references for JPA
@Resources({
@Resource(name="jdbc/ContactDS", type=DataSource.class),
@Resource(name="jdbc/ContactNoTxDS", type=DataSource.class)
})
@Stateless
public class ContactBean implements ContactLocal {

@PersistenceContext(unitName="ContactPU")
private EntityManager em;

@TransactionAttribute(SUPPORTS)
public List<Contact> findContactsByLastName(String lastName) {
 TypedQuery<Contact> q = em.createNamedQuery("ByLastName", Contact.class);
 q.setParameter("lastName", lastName);
 return q.getResultList();
}

@TransactionAttribute(REQUIRED)
public Contact saveContact(Contact c) {
 return em.merge(c);
}

@TransactionAttribute(REQUIRED)
public void clearContacts() {
 Query q = em.createNamedQuery("DeleteAll");
 q.executeUpdate();
}
}

ejbs/ContactLocal.java

package ejbs;

import java.util.List;
import javax.ejb.Local;
import entities.Contact;

@Local
public interface ContactLocal {
public List<Contact> findContactsByLastName(String lastName);
public Contact saveContact(Contact c);
public void clearContacts();
}

entities/Contact.java

package entities;

import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;
import javax.persistence.NamedQueries;
import javax.persistence.NamedQuery;

@Entity
@NamedQueries( {
@NamedQuery(name="ByLastName",
 query="SELECT c FROM Contact c WHERE c.lastName = :lastName"),
@NamedQuery(name="DeleteAll", query="DELETE FROM Contact c")
})
public class Contact {

@Id
@GeneratedValue
private Long id;

private String firstName;
private String lastName;
private String phoneNumber;
private String emailAddress;

public void setId(Long id) {
 this.id = id;
}

public Long getId() {
 return id;
}

public void setFirstName(String firstName) {
 this.firstName = firstName;
}

public String getFirstName() {
 return firstName;
}

public void setLastName(String lastName) {
 this.lastName = lastName;
}

public String getLastName() {
 return lastName;
}

public void setPhoneNumber(String phoneNumber) {
 this.phoneNumber = phoneNumber;
}

public String getPhoneNumber() {
 return phoneNumber;
}

public void setEmailAddress(String emailAddress) {
 this.emailAddress = emailAddress;
}

public String getEmailAddress() {
 return emailAddress;
}
}

META-INF/persistence.xml

 <?xml version="1.0"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.0"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
<persistence-unit name="ContactPU" transaction-type="JTA">
 <jta-data-source>java:comp/env/jdbc/ContactDS</jta-data-source>
 <non-jta-data-source>java:comp/env/jdbc/ContactNoTxDS</non-jta-data-source>
 <class>entities.Contact</class>
 <exclude-unlisted-classes>true</exclude-unlisted-classes>
 <properties>
   <!-- Create the database tables on startup -->
   <property name="openjpa.jdbc.SynchronizeMappings" value="buildSchema"/>
 </properties>
</persistence-unit>
</persistence>

The simple EJB + JPA module looks like this when loaded as an Eclipse project:


Since ContactModule uses only Java EE APIs, it only requires WAS_HOME\dev\JavaEE\j2ee.jar to compile. If you use WebSphere, OpenJPA, or other extensions you'll need to include additional jars in your build path. Remember, this is only to compile the EJB + JPA module. We'll need different jars to actually run the module in the Embeddable EJB Container.

Now that the module is defined, we need to create an EJB jar file. As of EJB 3.0, a deployment descriptor (ejb-jar.xml) is not required. You can either use the "jar" command to create the jar or use the Eclipse Export... -> Java -> JAR file wizard. Export the jar to a location such as C:\ContactModule.jar. The EJB jar file should contain:

• META-INF/MANIFEST.MF
• entities/Contact.class
• ejbs/ContactBean.class
• ejbs/ContactLocal.class
• META-INF/persistence.xml

We now have an EJB module. Traditionally, to test this module you'd write a web client, deploy it in the app server, and drive some http operations against it. Another option is to create a JEE client container app and then drive the EJB using launchClient. Or you may have used RAD's Universal Test Client to execute EJB operations. All of these options work, but they can be difficult to automate and some of them may require code modifications in order to work properly.

Let's take a look at how the Embeddable EJB Container can be used to simplify this testing using the popular JUnit 4 test framework. Again, I used an Eclipse project, but provided that you have the necessary JUnit 4 libraries, you can also run the same tests from the command line. To test the basic functions of our ContactBean session bean, we'll create a simple JUnit 4 annotated test class, TestContactBean.

test/TestContactBean.java

package test;

import java.util.List;

import javax.ejb.embeddable.EJBContainer;

import org.junit.Assert;
import org.junit.Test;

import ejbs.ContactLocal;
import entities.Contact;

public class TestContactBean {

@Test
public void testContactBean() {

 // Create test data
 Contact c1 = new Contact();
 c1.setFirstName("John");
 c1.setLastName("Smith");
 c1.setPhoneNumber("555-555-5555");
 c1.setEmailAddress("john@someplace.org");

 Contact c2 = new Contact();
 c2.setFirstName("Jane");
 c2.setLastName("Smith");
 c2.setPhoneNumber("555-555-5555");
 c2.setEmailAddress("jane@someplace.org");

 // Create the embeddable container
 EJBContainer ec = EJBContainer.createEJBContainer();

 try {
   // Use the embeddable container context to look up a bean
   ContactLocal contactBean = (ContactLocal) ec.getContext().lookup(
      "java:global/ContactModule/ContactBean!ejbs.ContactLocal");

   // Clear existing contacts
   contactBean.clearContacts();
  
   // Save new contacts
   contactBean.saveContact(c1);
   contactBean.saveContact(c2);

   // Query contacts and verify results
   List contacts = contactBean.findContactsByLastName("Smith");
   Assert.assertEquals(2, contacts.size());
   for (Contact c : contacts) {
     Assert.assertTrue(c.getLastName().equals("Smith"));
   }
 } catch (Throwable t) {
   Assert.fail("Caught unexpected exception: " + t.getMessage());
 } finally {
   ec.close();
 }
}
}

If you've used JPA in JSE-mode, the pattern looks pretty familiar, right? We use a static EJBContainer.createEJBContainer() method to create the embeddable container and then use a lookup to get an instance of the bean. Knowing what JNDI name to use can be a little tricky, but the new java:global namespace simplifies that as well. In general, you can use java:global/module-name/bean-name[/!interface] to look up the bean. After you have a reference to the bean, you can call its EJB methods. Piece of cake. Well there is a little more to do. In order to get the code to compile and run you need to include ContactModule.jar, WAS_HOME\runtimes\com.ibm.ws.ejb.embeddableContainer_8.0.0.jar, WAS_HOME\runtimes\com.ibm.ws.jpa.thinclient_8.0.0.jar, and the jUnit jars in your classpath.

If we were simply testing EJBs we'd be all set. JPA has a few more requirements we need to take care of. First, since we'll be using Derby as our database, we need to add the Derby library to our classpath. It can be found at WAS_HOME\derby\lib\derby.jar. Second, in an EE environment, JPA uses data sources to connect to a relational database. JPA typically requires both a transactional and non-transactional data source. WebSphere's embeddable container allows us to very easily configure data sources one of two ways. The most flexible of the two approaches is to configure them within a properties file. If a file named embeddable.properties is found on the classpath the properties in this file will automatically get loaded by the embeddable container . The WebSphere Embeddable EJB Container also allows you to pass properties in via Map on the createEJBContainer method. As a third option you can specify a system property to inform the embeddable container where its properties file resides. More information on the properties supported by the embeddable container can be found in the Information Center article Embeddable EJB container configuration properties. You'll find that the embeddable container is extremely configurable. You can even configure settings such as cache size, security options, and logging.

embeddable.properties

# JPA Transactional data source definition
DataSource.ds1.name=jdbc/ContactDS
DataSource.ds1.className=org.apache.derby.jdbc.EmbeddedXADataSource
DataSource.ds1.createDatabase=create
DataSource.ds1.databaseName=ContactDB
DataSource.ds1.transactional=true

# JPA non-Transactional data source definition
DataSource.ds2.name=jdbc/ContactNoTxDS
DataSource.ds2.className=org.apache.derby.jdbc.EmbeddedXADataSource
DataSource.ds2.createDatabase=create
DataSource.ds2.databaseName=ContactDB
DataSource.ds2.transactional=false

# Definitions for global to component namespace bindings - required for @Resource defs
Bean.#ContactModule#ContactBean.ResourceRef.BindingName.jdbc/ContactDS=jdbc/ContactDS
Bean.#ContactModule#ContactBean.ResourceRef.BindingName.jdbc/ContactNoTxDS=jdbc/ContactNoTxDS

Another approach is to define the data source using one or more @DataSourceDefinition(s) annotation(s) on the EJB. For example:

@DataSourceDefinitions({
@DataSourceDefinition(name="jdbc/ContactDS",
 className="org.apache.derby.jdbc.EmbeddedXADataSource",
 databaseName="JPAinEmbeddedContainer",
 transactional=true,
 properties={"createDatabase=create"}),
@DataSourceDefinition(name="jdbc/ContactNoTxDS",
 className="org.apache.derby.jdbc.EmbeddedXADataSource",
 databaseName="JPAinEmbeddedContainer",
 transactional=false,
 properties={"createDatabase=create"})
})
@Stateless
public class ContactBean implements ContactLocal {
...
}

As you can see, using @DataSourceDefinition is a very simple way to define data sources. There is no properties file or @Resource references to manage. The data sources automatically get registered into the java:comp/env namespace (take a look jta-data-source and non-jta-data-source definitions in the persistence.xml - as a best practice, component data sources are used). However, simplicity comes at the cost of flexibility. A change in the data source definition requires recompiling the application.


There's one more JPA-specific item we need to take care of before running the test. In the full application server, the container takes care of JPA entity enhancement. This does not occur in the embeddable container. Fortunately, you can either enhance the entities at build time or the JPA agent-based enhancer can be used to perform the enhancement at runtime. Enabling the enhancer is as simple as specifying the JVM option -javaagent:C:\/"Program Files/"\IBM\WebSphere\AppServer\runtimes\com.ibm.ws.jpa.thinclient_8.0.0.jar on the java or Eclipse jUnit invocation. If all goes well, you should see a result similar to this...



More information about the Embeddable EJB Container is available in the WebSphere Application Server V8 Information Center. Happy testing!

-Jeremy

Type-safe JPQL query results with JPA 2.0

If you use the Eclipse IDE or Rational Application Developer (RAD) for your Java development, you probably make an effort to keep an eye out for warnings flagged in your code and spend some amount of time cleaning them up. That is why they are there, right? To help make sure your code is top notch. Warnings are even more prevalent if you have a large amount of pre-Java 5.0 code and have moved to Java 5 or Java 6. If you had a fairly warning-free code base and moved to Java 5 or newer you've probably encountered a barrage of new warnings in your workspace. The introduction of Java generics in Java 5.0 is one of the major culprits for all these new warnings. The use of collection classes (Collection, Set, List, etc.) that you happily used in your code are now all warning flagged as needing to be parameterized/typed.

If you are a user of JPA, normal usage of some of the JPA APIs (primarly those from JPA 1.0) will add to your warning woes. For example, result list processing of JPQL queries returns an untyped result list that will get flagged as needing type information. Let's say you have some code that creates a JPQL query, executes it, and prints out the result.

Query q = em.createQuery("SELECT e FROM Employee e");
List employees = q.getResultList();
if (employees != null) {
  for (int i = 0; i < employees.size(); i++) {
    Employee emp = (Employee)employees.get(i);
    System.out.println("Employee id=" + emp.getId() + ", name=" + e.getName());
  }
}

The Eclipse IDE will flag the 2nd line as a warning, telling you that you need to parameterize the List. A quick and dirty way to get rid of the warning is to wildcard the List definition, informing the compiler the list is untyped.

List<?> employees = q.getResultList(); // Good-bye, warning!

This makes a dent in your warning count, but doesn't really buy you much. You are left with potentially unsafe type casts and result processing is still a bit messy. This is where a new feature/interface of JPA 2.0, TypedQuery shines. The TypedQuery interface provides the ability to operate on query results and parameters in a more type-safe manner. The TypedQuery interface is actually geared more toward the new programmatic Criteria API, but provides benefits for JPQL queries as well. Here's an example of how one might write the original code using a TypedQuery.

TypedQuery<Employee> tq = em.createQuery("SELECT e FROM Employee e",     Employee.class);
for (Employee emp : tq.getResultList()) {
  System.out.println("Employee id=" + emp.getId() + ", name=" + e.getName());
}

The compile time warnings are gone, the casts are gone, and we are able to use the new iterator-based for loop to cleanly process the query result. Looks great, right? Well, there is a caveat. Currently, JPQL-based TypedQuery does not support the use of the JPA 2.0 Tuple to process multi-valued results (for example, returning multiple entity types in a single "row" of the result). However, the JPA 2.0 Criteria API does provide full Tuple support - along with all around true type-safety for query, query result, and query parameters. With any luck the ability to use Tuple with JPQL may make its way in a future version of the JPA specification. But, if your JPQL queries primarily return a single value type, the TypedResult support for JPQL currently provided in JPA 2.0 could go a long way toward writing cleaner code.

-Jeremy

Using Bean Validation with the OSGi/JPA 2.0 Feature Pack

Do you have applications littered with layers of inconsistent validation logic or are writing a new application that will require data validation? Bean validation (JSR-303 - part of the Java EE6 family of specifications) defines an API for providing runtime validation of data within Java beans. In addition, other EE6 specifications such as JPA 2.0, JSF, and JCA provide tight integration with bean validation, allowing runtime validation to occur seemlessly with your persistence, resource/connectivity, and UI logic.

Using bean validation within the app server requires a JSR-303 spec API library, the bean validation provider library(ies), and any supporting libraries. If you'd like to test drive bean validation with JPA 2.0, the WebSphere Application Server V7 Feature Pack for OSGi Applications and Java Persistence API 2.0 does not include a bean validation API or provider, but it does include a provision for wiring one in yourself. The wiring is pretty simple and is briefly explained in this IBM Infocenter document Using Bean Validation with JPA. With the information provided in the article, I was able to configure WebSphere Application Server to use Apache's Bean Validation provider using these steps.

1. Install WebSphere Application Server V7 with the Fixpack required for the feature pack (currently 7.0.0.9). There is a free Developer's Edition if you don't have the V7 test server for RAD 7.5 or an extra license lying about. :-)
2. Install the WebSphere Application Server V7 Feature Pack for OSGi Applications and Java Persistence API 2.0.
3. Download the Apache Bean Validation Spec API jar. Save to a location such as:
   /opt/apache/specs/geronimo-validation_1.0_spec-1.1.jar
4. Download the Apache Bean Validation Core jar. Save to a location such as:
   /opt/apache/bval/bval-core-0.1-incubating.jar
5. Download the Apache Bean Valdation JSR-303 jar. Save to a location such as:
   /opt/apache/bval/bval-jsr303-0.1-incubating.jar
6. Download Apache Commons Bean Utils. Extract the distribution and locate commons-beanutils-1.8.3.jar. Copy the jar to a location such as:
   /opt/apache/commons/commons-beanutils-1.8.3.jar
7. Download Apache Commons Lang 2.4. Extract the distribution and locate commons-lang-2.4.jar. Copy the jar to a location such as:
   /opt/apache/commons/commons-lang-2.4.jar
8. Start the app server.
9. Open the WebSphere Integrated Solutions Console in your browser (http://<server:port>/admin).
10. In the navigation pane expand Servers -> Server Types and click on WebSphere application servers.
11. Select the server profile to configure. Typically server1 in a default single server environment.
12. Under the Server Infrastructure section, select Process Definition.
13. Under the Additional Properties section, select Java Virtual Machine.
14. Under the Additional Properties section of the JVM settings, select Custom properties.
15. Click New and add a property named com.ibm.websphere.validation.api.jar.path and set the value to the location of the bean validation API jar. (ex. /opt/apache/specs/geronimo-validation_1.0_spec-1.1.jar)
16. Restart the application server.

The application server is now configured to locate and load the bean validation spec API jar. The spec API classes are now available, but the application server (being an EE5 app server) does include a bean validation provider. You need to make a provider available to your application by either bundling it with your application archive (in the proper location within a war or ear) or via a shared library. Include the bean validation core, bean validation jsr-303 component, commons bean utils, and commons lang 2.4 jars within the application or shared library and you'll be off to the races.

That's all there is to configuring the application server. To actually build applications which take advantage of bean validation constraints, you'll need to specify the spec API library in the build path of your tooling. The examples in Apache Bean Validation Primer will help you get started. The example in the primer is targeted for a JSE environment (I'm working on a JEE version -- no ETA), but the JPA entities, constraints, validators, and logic can be easily ported to an EE environment. One additional very important item to note is that your persistence.xml must be updated to version 2.0 in order for bean validation to be enabled by the JPA provider (OpenJPA). Since the 2.0 schema is backward compatible, it is typically a simple one line XML change. The JPA Integration section of the primer contains the rationale for this requirement and an example.

-Jeremy

OpenJPA OpenBooks sample available for WebSphere Application Server

A JEE version of OpenJPA's OpenBooks sample is now available for WebSphere Application Server with the WebSphere Application Server V7 Feature Pack for OSGi Applications and Java Persistence API 2.0. OpenBooks is a sample application that highlights many of the new features of JPA 2.0. The sample can be built from source obtained from the Apache's OpenJPA svn repository. If you are not familiar with svn, don't let it deter you from getting the sample. Use of svn for basic functions, like checking out a source tree, is a piece of cake. If you are in a single server environment, the OpenBooks build scripts (run with WebSphere's ws_ant) will compile, package, install, and configure OpenBooks. The OpenBooks instructions will help you get going. In particular, take a look at the section Deploy OpenBooks in an Application Server. This section contains some WebSphere-specific instructions.

The build scripts are primarily targeted for a single server environment, but may work in an ND environment as well. By default, OpenBooks is installed to the default server and port and the script assumes admin security is disabled. If you need to target a specific server and/or have security enabled, there are command line variables you can use to specify these details. The script is currently all or nothing, so if you specify one, you must specify all. For example:

$ /opt/WebSphere/AppServer/bin/ws_ant -Dbuild.mode=jee -Dappserver=was -Dwas.home=/opt/WebSphere/AppServer -Dwas.server=localhost -Dwas.port=8880 -Dwas.conntype=SOAP -Dwas.user=myuser -Dwas.password=mypassword

This invocation will install OpenBooks on the local machine with the profile serviced by SOAP admin port 8880 with the specified credentials. The script creates a new Derby data source (which is also configured to create a new database if it doesn't exist) by default, but if you are data source savvy you can easily configure the OpenBooks Data Source and OpenBooks Non-transactional Data Source to point at your own database. Assuming your build and install was successful, point your browser at http://<server>:<port>/OpenBooks to run the sample. Enjoy!!

-Jeremy

Apache OpenJPA 2.0.0 Now Available!

I am proud to announce the availability of Apache OpenJPA 2.0.0!

This distribution is based on the final JSR 317 Java Persistence API, Version 2.0 specification and passes the JPA 2.0 TCK, while remaining backwards compatible with prior releases based on the Java Persistence API (JPA 1.0) part of Java Community Process JSR-220 (Enterprise JavaBeans 3.0).

The WebSphere JPA solution is based on the Apache OpenJPA project.

Check it out!

Kevin

Beta 2 of Apache OpenJPA now available!

As we continue to march towards a 2.0.0 release of OpenJPA, we recently announced a Beta 2 of the OpenJPA 2.0.0 deliverable. This release contains additional bug fixes and performance improvements in addition to continuing our TCK compliance.

Enjoy! Let us know your views and comments either via this forum or the OpenJPA mailing lists.

Thanks,
Kevin

WebSphere V7 Feature Pack for OSGi Applications and Java Persistence API (JPA) 2.0 Open Beta now Available!

I actually have two announcements to make. From a WebSphere perspective, I am pleased to announce the availability of the IBM WebSphere Application Server V7 Feature Pack for OSGi Applications and Java Persistence API (JPA) 2.0 Open Beta.

I know that's a mouth full, but this announcement clearly demonstrates IBM's commitment to both the OSGi and JPA programming models. Each of these features can be individually installed on a WebSphere v7 system (specifically, 7.0.0.7). More details on the specifics can be found on the official announcement page above.

The other item I would like to announce is the availability of the Apache OpenJPA 2.0.0-beta release. This beta release is 100% compatible with the JPA 2.0 TCK, plus it contains several other improvements outlined on the announcement. This OpenJPA 2.0.0-beta has been available for a couple of weeks, but I was waiting to announce it in conjunction with the WebSphere Open Beta. (Apache OpenJPA provides the basis for the WebSphere JPA solution.)

So, whether you are a WebSphere customer or just interested in experimenting with the latest JPA 2.0 feature set, one of these two offerings should fit the bill. Enjoy!

Basic tutorial for OpenJPA

Take a look at this new tutorial on the Apache site.

Begin using OpenJPA - The Basics
This sample and tutorial steps through the requirements, setup, code and deployment of an OpenJPA implementation with a JSP web application on Geronimo. It is geared for those that are just beginning with OpenJPA. Users will be given detailed instructions for each part of the setup including the setup of Eclipse, Geronimo and the Derby database. For those developers that have been wanting to explore the power of OpenJPA, this is where you begin.

This will get someone going in OpenJPA even if they are new to Geronimo, Derby and OpenJPA

OpenJPA tutorial for beginners available

With the release of the IBM WebSphere OpenJPA V2.0 open Alpha, there will be a renewed interest in OpenJPA by those that have not yet learned it. For those that have been using OpenJPA, this release will provide new functions and features. For those that have never used JPA, the Open Alpha site has an excellent getting-started tutorial. This tutorial is a great place to learn the basics. This tutorial is labeled as part 1 and does not require the WebSphere Application Server. It uses J2SE, MySQL and Eclipse.
The tutorial steps through the parts of OpenJPA, including the way it uses annotations, the persistence.xml file and JPQL. To keep it simple and to allow the user maximum flexibility the complete working code is provided.

You can download the tutorial here:
IBM Websphere OpenJPA Tutorial Part 1
*free site registration required

WebSphere JPA 2.0 Open Alpha is now available!

Hi,
The WebSphere JPA team is proud to announce the availability of the IBM WebSphere Application Server V7 Java Persistence API (JPA) 2.0 Open Alpha. Details can be found here...

https://www14.software.ibm.com/iwm/web/cc/earlyprograms/websphere/wsasjpaoa/

This JPA 2.0 Open Alpha is based off the Public Final Draft #2 of the JPA 2.0 specification, which can be found here...

http://jcp.org/en/jsr/detail?id=317

If you are looking for something a bit more readable that highlights the new JPA 2.0 features, you could also reference the following two articles...

http://www.ibm.com/developerworks/websphere/techjournal/0909_col_sutter/0909_col_sutter.html
http://www.ibm.com/developerworks/java/library/j-typesafejpa/

Since the WebSphere JPA solution is built on top of the Apache OpenJPA project, you can always follow our progress via these URLs...

http://openjpa.apache.org/
http://openjpa.apache.org/jpa-20-roadmap.html

Enjoy!
Kevin

OpenJPA with solidDB

Donald Vines and myself recently published a DeveloperWorks article on how to use solidDB with OpenJPA. In a nutshell, since solidDB is a JDBC-compliant in-memory database, this configuration just worked out-of-the-box with the generic JDBC dictionary provided by OpenJPA. It still is on our wish list to create a specific dictionary for solidDB, but in the mean time, this article shows that the integration of solidDB and OpenJPA is possible today.

http://www.ibm.com/developerworks/websphere/techjournal/0906_vines/0906_vines.html

Enjoy!
Kevin

Intro to OpenJPA Caching

Intro to OpenJPA Caching

A part of the JPA 2.0 spec (JSR317) defines the javax.persistence.Cache Interface [1] which exposes a providers' second level(L2) cache. While exploring this new interface, I learned a few things about OpenJPA caching that were not entirely obvious to me at the time. Hopefully I can lay out some of what I've learned to save at least one person some pain.

First off, I'm going to discuss the caching that wasn't obvious to me. Take a quick look through the semi-pseudo code that models the scenario I was running below.

EntityManager em = EntityManagerFactory.createEntityManager()
Entity e = em.findEntity(Entity.class, Long.valueOf(1))
updateEntityViaJDBC(e.getId(), "new data")// a worker method to insert data using JDBC
e = em.findEntity(Entity.class, Long.valueOf(1))
if(e.getData().equals("new data")==false){
//Whoops, where did my updated data go!?!
}

As you can see in the example above, I find an Entity from the database and then update the database using JDBC. Since my database was updated I figured I needed search again to update my Entity and finally I validated that the Entity was holding onto the data I *thought* it should. The last part of my logic is where I went astray.

Once an Entity is loaded by OpenJPA, it is characterized as a managed Entity. When an Entity is managed by the JPA runtime, the spec says that "Synchronization to the database does not involve a refresh of any managed entities unless the refresh operation is explicitly invoked on those entities" [2]. The more I dug into this one, I found that OpenJPA tries to cache/optimize where ever the spec allows. If you were to create an EntityManager and then call em.find() on the same object 1000 times in a row, OpenJPA would only hit the DB once. I didn't expect that to happen, but I can swallow it now that I know that it is happening! This caching is sometimes referred to as the EntityManager L1 cache and it is scoped to the life of an EntityManager. In short, when an EntityManger falls out of scope or is closed, the L1 cache is cleared. I'm not going to lie, this stuff is complicated and I only discussed a small part of the entire picture. If you want/need more details, please see [3] for the entire description.

One thing to note is that the previous paragraph talked only about the EntityManager L1 cache which is defined by the spec and it shouldn't be confused with the following paragraph which pertains to the L2 cache.

The javax.persistence.Cache interface that is being introduced as part of JSR317 essentially exposes some of the functionality from org.apache.openjpa.persistence.StoreCache that has been in existence since the early days of OpenJPA. The interface itself is not that interesting, but the results from enabling the OpenJPA data cache are pretty impressive. The OpenJPA user manual states that "This cache is designed to significantly increase performance while remaining in full compliance with the JPA standard. This means that turning on the caching option can transparently increase the performance of your application, with no changes to your code." I hate to say this, but it is a case of where you can get something for free. Enabling the data cache is as simple as adding the following property to your persistence.xml.

< name="openjpa.DataCache" value="true">

For more information regarding the OpenJPA data cache see [4].

-Rick

http://jcp.org/aboutJava/communityprocess/pfd/jsr317/index.html --JSR317 download page.
[1] See 6.10 of JSR317.
[2] See 3.2.4 of JSR317.
[3] Chapter 6 of JSR317.
[4] http://openjpa.apache.org/builds/1.0.2/apache-openjpa-1.0.2/docs/manual/ref_guide_caching.html

Custom ORM with OpenJPA

While JPA provides a very rich set of mapping constructs, it doesn't offer much in terms of customization in situations where a domain model cannot be directly or easily mapped to the database using those constructs. This is more typically an issue when an existing application is being converted from some persistence mechanism such as a JDBC-based DAO to JPA and its domain classes cannot suffer more than minimal modifications.

For example, let's say there is an application XYZ which has a domain class named User. The User class contains an email address field which is of type string. Application XYZ expects email addresses in the form user@domain. However, the database table backing the User class stores the components of an email address in separate columns, EMAIL_USER and EMAIL_DOMAIN. The persistence mechanism used in XYZ (for example, JDBC) performs the necessary transforms in a DAO layer to compose the database columns to a single string value of the expected format and vice versa.

Converting this application to JPA would typically involve mapping the individual persistent fields for user and domain to the individual database columns and then wrappering them with the logic formerly provided by the DAO to do the transforms. While this is fairly straight forward, it requires modifications to the domain object and couples the transform logic to the entity. These types of changes can be invasive and undesirable in an existing application.

OpenJPA has an elegant solution for handling these types of mappings: mapping strategies. Custom mapping strategies can be applied to persistent fields using OpenJPA’s @org.apache.openjpa.persistence.jdbc.Strategy annotation. Strategies are most simply created by subclassing the org.apache.openjpa.jdbc.meta.strats.AbstractValueHandler class. Although, for complex mappings you may decide to provide a full implementation class by implementing the org.apache.openjpa.jdbc.meta.ValueHandler interface. To implement a basic strategy you simply need to provide column mapping information and the methods to transform to and from a given type. The name of the custom strategy class is specified on the @Strategy annotation. At runtime, OpenJPA applies this strategy to the persistent field or property. This decouples the mapping and data transformation logic from the domain class. Below is a simple strategy for the multi-column email address mapping. It maps a single persistent field to two database columns and provides the transforms to get the data in the proper format to/from the domain model and database table.

EmailAddressStrategy.java

package strategies;

import org.apache.openjpa.jdbc.kernel.JDBCStore;
import org.apache.openjpa.jdbc.meta.ValueMapping;
import org.apache.openjpa.jdbc.meta.strats.AbstractValueHandler;
import org.apache.openjpa.jdbc.schema.Column;
import org.apache.openjpa.jdbc.schema.ColumnIO;
import org.apache.openjpa.meta.JavaTypes;

public class EmailAddressStrategy extends AbstractValueHandler {

  // Create the custom column mappings for this strategy
  public Column[] map(ValueMapping vm, String name, ColumnIO io,
    boolean adapt) {

    Column user = new Column();
    user.setName("EMAIL_USER");
    user.setJavaType(JavaTypes.STRING);

    Column domain = new Column();
    domain.setName("EMAIL_DOMAIN");
    domain.setJavaType(JavaTypes.STRING);
    return new Column[]{ user, domain };
  }

  // Transform the email address value to its individual datastore column values.
  public Object toDataStoreValue(ValueMapping vm, Object val,
    JDBCStore store) {
    if (val == null)
      return null;

    // Split the user and domain components into distinct values
    if (val instanceof String) {
      String strVal = (String)val;
      String user = strVal;
      String domain = null;
      int atIndex = user.indexOf('@');
      if (atIndex != -1) {
        user = strVal.substring(0, atIndex);
        domain = strVal.substring(atIndex+1);
      }
      return new Object[] { user, domain };
    }
    return val.toString();
  }

  // Transform the separate datastore values to an email address.
  public Object toObjectValue(ValueMapping vm, Object val) {
    if (val == null)
      return null;

    if (val instanceof Object[]) {
      String user = null;
      String domain = null;
      Object[] objArray = (Object[])val;
      if (objArray.length > 0)
        user = objArray[0].toString();
      if (objArray.length > 1)
        domain = objArray[1].toString();
     return user + "@" + domain;
    }
    return val.toString();
  }
}

User.java

import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;

import org.apache.openjpa.persistence.jdbc.Strategy;

@Entity
public class User {

  @Id
  @GeneratedValue
  private int id;

  // Use email address strategy on this persistent field
  @Strategy("strategies.EmailAddressStrategy")
  private String emailAddress;

  public void setEmailAddress(String emailAddress) {
    this.emailAddress = emailAddress;
  }

  public String getEmailAddress() {
    return emailAddress;
  }
}

In short, if you find that JPA's mapping constructs are not quite adequate for a particular mapping, creating your own mapping strategy may be just the ticket.

-Jeremy

What's in a word?

I wanted to understand what issues are important to the growing base of OpenJPA users. So I attempted a small experiment as follows:

• Extracted the headers from the users' postings in OpenJPA Users' Mailing List.
• The extraction process was a breeze with the help of HTML Parser. The program found a list of 1059 independent poster headers.
• Fed the headers to Wordle to create a Word Cloud.

This is the Word Cloud after the first attempt

 

The original Word Cloud is here 

Of course, "problem" is often what makes a user to post in the forum. To focus further and because did not like "problem" being so prominent in the Word Cloud, removed the following words for the input list of headers: "OpenJPA", "entity", "mapping", "error", "using", "Re:"

The original Word Cloud is here

OpenJPA Enhancement Eclipse Builder

As I wrote about in a previous blog post, enhancement is HIGHLY recommended when developing an application that uses OpenJPA. The OpenJPA community has been vigilant about trying to cover every enhancement scenario, but after my last posting a reader brought up new somewhat uncovered situation. I say somewhat, because I'm sure the user could have enhanced using one of the existing methods, but it would not have been pretty. My understanding is that they are using Eclipse WTP to write and deploy their application, and orm.xml to map their Entities. We have the first two variants covered with an Eclipse plugin, but the third is something that we have not addressed. I did some research and found that creating a custom Ant builder may be the perfect fit. This builder will be executed by Eclipse after compilation has completed. Below I've documented the steps required to create an Ant builder that will enhance your Entities after each compile.

Please contact me with any questions/comments.

-Rick

For steps documented below, I have the following directory structure. Note, these steps must be followed for each project that has Entities that need to be enhanced.

/builder_project
enhance.xml <- the OpenJPA builder.... download here.
/bin <- Compile directory
/src <- Source directory
/jpa_lib <- OpenJPA binary and all jars from the lib dir of the binary download
commons-collections-3.2.jar
commons-lang-2.1.jar
commons-pool-1.3.jar
derby-10.2.2.0.jar
geronimo-jpa_2.0_spec-1.0-EA-SNAPSHOT.jar
geronimo-jta_1.1_spec-1.1.1.jar
openjpa-2.0.0-SNAPSHOT.jar
serp-1.13.1.jar
/lib <- other libs

1. After you add the enhance.xml file to your file system, make sure to refresh your Eclipse workspace so it knows about the newly added file. Make sure that the enhance.xml file is listed in the Navigator view.
   
2. Right click on the Eclipse project that you want to enhance and click on properties.
3. Click on the builders filter, and create a new Ant builder.
4. Name your builder, then click on "Browse Workspace" in the buildfile box. If you downloaded the enhance.xml file and refreshed your workspace, it should be listed there. If not, go back to step 1 and make sure that Eclipse detects your enhance.xml file.
   
5. In the "Base Directory" box, click on the variables button and select build_project. This should refer to the root of your project. In the directory structure above, it refers to "builder_project".
   
6. In the "Arguments" box you need to add the following properties -Dopenjpa.libs and -Dbuild.dir. -Dopenjpa.libs is the path to the OpenJPA libs, relative to the root of the project. -Dbuild.dir is the path to the build directory, relative to the root of the project. In the directory structure above, openjpa.libs should be set to jpa_lib and build.dir should be set to bin.
   
7. Click on the "Targets" tab along the top.
8. You need to set the enhance target to run as a part of "Manual Build" and "Auto Build".
   

Screenshots:

Files:
[1]http://filebin.ca/zbfbg/enhance.xml -- enhance.xml
[2]http://filebin.ca/aayeg/navigator.png -- Navigator pane
[3]http://filebin.ca/cznvmv/main.png -- Edit builder configuration - main
[4]http://filebin.ca/fqnadv/targets.png -- Edit builder configuration - targets

Jazz and Software Development

Jazz is said to be the fundamental rhythms of human life and man’s contemporary reassessment of his traditional values. The ancient beats and rhythms of Africa found a contemporary expression in the new world of New Orleans around early twentieth century. Often music historians spoke about undefinable quality of Jazz. Louis Armstrong -- one of the greatest Jazz musician ever - put is succinctly : "If you gotta ask, you’ll never know". The unscripted collaboration where every man plays his own tune but responds and respects others' -- is the core theme that gave rise to this uniquely American contribution to music.

It is intriguing that the new software collaboration platform built on Eclipse from contributions by pioneers such as Erich Gamma is named Jazz. Naming is very important. Naming captures aspiration. A software engineer as great as Erich Gamma can foresee the evolution of software development practices or at least where he wants it to go. Building software is playing Jazz -- not Orchestra.

Dynamic Fetch Planning

Now I have heard this before:

    "Hibernate can do X, can your thing do X?"

I mumble in a me-too manner. Because Hibernate, as the world knows, is 42 -- the answer to the eternal question. Also I have seen many postings in OpenJPA's own mailing list that can be paraphrased as: "We are migrating our Hibernate application to OpenJPA, but we could do X with Hibernate, OpenJPA does not seem to be able to do X". Or a slight variation on the theme: "This test fails with OpenJPA. But it passes with Hibernate".

As a contributor to OpenJPA, this implied second-class status irks me at times (I also wonder: why are they migrating away from Hibernate anyway?). So when the following email from an unknown sender reached my mailbox -- I smiled. The mail says:

"We are working in investment banking's IT sector. We have a requirement X that Hibernate can not support. Can OpenJPA do X?"

Firstly, I was happy to note that OpenJPA can do X very well.

Secondly, the proposal to include feature X to JPA 2.0 Specification was simply ignored.

Thirdly, investment banking's IT is the bellwether of enterprise technology trends and demands.

I must admit that I do not know Hibernate deep enough to ascertain that it can or can not do X -- I am just going by the sender's comments.

So what is this feature X? It is called Dynamic FetchPlan.

The conventional wisdom is to mix fetch plan with query. This wisdom is rooted in the history of SQL -- where projection and selection criteria appear together in an all-encompassing SQL statement (I have heard stories of a single SQL statement being 2000 lines long). JPA took the same route -- its query language JPQL introduced FETCH JOIN which lets you specify what instances (that are not directly part of the projected result) will be brought in the memory as a side-effect of query execution.

While OpenJPA supports fetch join as per JPA specification -- OpenJPA prefers to separate these two concerns, namely: i) what is selected and ii) what is fetched. Because fetch plan is independently specified, you can issue a simple EntityManager.find(Company.class, "acme.org") -- and as a result the persistent context may either  be populated with a single Company instance or with all the Departments and all the Employees of those Departments with their Addresses and Spouses' names. It all depends on what fetch plan is active while you invoked EntityManager.find(). In one end of the spectrum, you can use basic default fetch plan which fetches the fields of primitive types but no relations. On the other end, you can be as creative as you wish to specify a sub-graph of the complete closure starting from a root Company instance. And you can specify these fetch plans during design, use them and edit them at runtime.

But why should you care to carve out a sub-graph from a root entity? Why does OpenJPA use fetch plan as a pervasive notion throughout its internal design? Why even the question may be worth pondering?

My take on it will take a separate post -- more importantly Oscar is starting in TV....