OpenJPA with solidDB

Donald Vines and myself recently published a DeveloperWorks article on how to use solidDB with OpenJPA. In a nutshell, since solidDB is a JDBC-compliant in-memory database, this configuration just worked out-of-the-box with the generic JDBC dictionary provided by OpenJPA. It still is on our wish list to create a specific dictionary for solidDB, but in the mean time, this article shows that the integration of solidDB and OpenJPA is possible today.

http://www.ibm.com/developerworks/websphere/techjournal/0906_vines/0906_vines.html

Enjoy!
Kevin

Intro to OpenJPA Caching

Intro to OpenJPA Caching

A part of the JPA 2.0 spec (JSR317) defines the javax.persistence.Cache Interface [1] which exposes a providers' second level(L2) cache. While exploring this new interface, I learned a few things about OpenJPA caching that were not entirely obvious to me at the time. Hopefully I can lay out some of what I've learned to save at least one person some pain.

First off, I'm going to discuss the caching that wasn't obvious to me. Take a quick look through the semi-pseudo code that models the scenario I was running below.

EntityManager em = EntityManagerFactory.createEntityManager()
Entity e = em.findEntity(Entity.class, Long.valueOf(1))
updateEntityViaJDBC(e.getId(), "new data")// a worker method to insert data using JDBC
e = em.findEntity(Entity.class, Long.valueOf(1))
if(e.getData().equals("new data")==false){
//Whoops, where did my updated data go!?!
}

As you can see in the example above, I find an Entity from the database and then update the database using JDBC. Since my database was updated I figured I needed search again to update my Entity and finally I validated that the Entity was holding onto the data I *thought* it should. The last part of my logic is where I went astray.

Once an Entity is loaded by OpenJPA, it is characterized as a managed Entity. When an Entity is managed by the JPA runtime, the spec says that "Synchronization to the database does not involve a refresh of any managed entities unless the refresh operation is explicitly invoked on those entities" [2]. The more I dug into this one, I found that OpenJPA tries to cache/optimize where ever the spec allows. If you were to create an EntityManager and then call em.find() on the same object 1000 times in a row, OpenJPA would only hit the DB once. I didn't expect that to happen, but I can swallow it now that I know that it is happening! This caching is sometimes referred to as the EntityManager L1 cache and it is scoped to the life of an EntityManager. In short, when an EntityManger falls out of scope or is closed, the L1 cache is cleared. I'm not going to lie, this stuff is complicated and I only discussed a small part of the entire picture. If you want/need more details, please see [3] for the entire description.

One thing to note is that the previous paragraph talked only about the EntityManager L1 cache which is defined by the spec and it shouldn't be confused with the following paragraph which pertains to the L2 cache.

The javax.persistence.Cache interface that is being introduced as part of JSR317 essentially exposes some of the functionality from org.apache.openjpa.persistence.StoreCache that has been in existence since the early days of OpenJPA. The interface itself is not that interesting, but the results from enabling the OpenJPA data cache are pretty impressive. The OpenJPA user manual states that "This cache is designed to significantly increase performance while remaining in full compliance with the JPA standard. This means that turning on the caching option can transparently increase the performance of your application, with no changes to your code." I hate to say this, but it is a case of where you can get something for free. Enabling the data cache is as simple as adding the following property to your persistence.xml.

< name="openjpa.DataCache" value="true">

For more information regarding the OpenJPA data cache see [4].

-Rick

http://jcp.org/aboutJava/communityprocess/pfd/jsr317/index.html --JSR317 download page.
[1] See 6.10 of JSR317.
[2] See 3.2.4 of JSR317.
[3] Chapter 6 of JSR317.
[4] http://openjpa.apache.org/builds/1.0.2/apache-openjpa-1.0.2/docs/manual/ref_guide_caching.html

Custom ORM with OpenJPA

While JPA provides a very rich set of mapping constructs, it doesn't offer much in terms of customization in situations where a domain model cannot be directly or easily mapped to the database using those constructs. This is more typically an issue when an existing application is being converted from some persistence mechanism such as a JDBC-based DAO to JPA and its domain classes cannot suffer more than minimal modifications.

For example, let's say there is an application XYZ which has a domain class named User. The User class contains an email address field which is of type string. Application XYZ expects email addresses in the form user@domain. However, the database table backing the User class stores the components of an email address in separate columns, EMAIL_USER and EMAIL_DOMAIN. The persistence mechanism used in XYZ (for example, JDBC) performs the necessary transforms in a DAO layer to compose the database columns to a single string value of the expected format and vice versa.

Converting this application to JPA would typically involve mapping the individual persistent fields for user and domain to the individual database columns and then wrappering them with the logic formerly provided by the DAO to do the transforms. While this is fairly straight forward, it requires modifications to the domain object and couples the transform logic to the entity. These types of changes can be invasive and undesirable in an existing application.

OpenJPA has an elegant solution for handling these types of mappings: mapping strategies. Custom mapping strategies can be applied to persistent fields using OpenJPA’s @org.apache.openjpa.persistence.jdbc.Strategy annotation. Strategies are most simply created by subclassing the org.apache.openjpa.jdbc.meta.strats.AbstractValueHandler class. Although, for complex mappings you may decide to provide a full implementation class by implementing the org.apache.openjpa.jdbc.meta.ValueHandler interface. To implement a basic strategy you simply need to provide column mapping information and the methods to transform to and from a given type. The name of the custom strategy class is specified on the @Strategy annotation. At runtime, OpenJPA applies this strategy to the persistent field or property. This decouples the mapping and data transformation logic from the domain class. Below is a simple strategy for the multi-column email address mapping. It maps a single persistent field to two database columns and provides the transforms to get the data in the proper format to/from the domain model and database table.

EmailAddressStrategy.java

package strategies;

import org.apache.openjpa.jdbc.kernel.JDBCStore;
import org.apache.openjpa.jdbc.meta.ValueMapping;
import org.apache.openjpa.jdbc.meta.strats.AbstractValueHandler;
import org.apache.openjpa.jdbc.schema.Column;
import org.apache.openjpa.jdbc.schema.ColumnIO;
import org.apache.openjpa.meta.JavaTypes;

public class EmailAddressStrategy extends AbstractValueHandler {

  // Create the custom column mappings for this strategy
  public Column[] map(ValueMapping vm, String name, ColumnIO io,
    boolean adapt) {

    Column user = new Column();
    user.setName("EMAIL_USER");
    user.setJavaType(JavaTypes.STRING);

    Column domain = new Column();
    domain.setName("EMAIL_DOMAIN");
    domain.setJavaType(JavaTypes.STRING);
    return new Column[]{ user, domain };
  }

  // Transform the email address value to its individual datastore column values.
  public Object toDataStoreValue(ValueMapping vm, Object val,
    JDBCStore store) {
    if (val == null)
      return null;

    // Split the user and domain components into distinct values
    if (val instanceof String) {
      String strVal = (String)val;
      String user = strVal;
      String domain = null;
      int atIndex = user.indexOf('@');
      if (atIndex != -1) {
        user = strVal.substring(0, atIndex);
        domain = strVal.substring(atIndex+1);
      }
      return new Object[] { user, domain };
    }
    return val.toString();
  }

  // Transform the separate datastore values to an email address.
  public Object toObjectValue(ValueMapping vm, Object val) {
    if (val == null)
      return null;

    if (val instanceof Object[]) {
      String user = null;
      String domain = null;
      Object[] objArray = (Object[])val;
      if (objArray.length > 0)
        user = objArray[0].toString();
      if (objArray.length > 1)
        domain = objArray[1].toString();
     return user + "@" + domain;
    }
    return val.toString();
  }
}

User.java

import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;

import org.apache.openjpa.persistence.jdbc.Strategy;

@Entity
public class User {

  @Id
  @GeneratedValue
  private int id;

  // Use email address strategy on this persistent field
  @Strategy("strategies.EmailAddressStrategy")
  private String emailAddress;

  public void setEmailAddress(String emailAddress) {
    this.emailAddress = emailAddress;
  }

  public String getEmailAddress() {
    return emailAddress;
  }
}

In short, if you find that JPA's mapping constructs are not quite adequate for a particular mapping, creating your own mapping strategy may be just the ticket.

-Jeremy

What's in a word?

I wanted to understand what issues are important to the growing base of OpenJPA users. So I attempted a small experiment as follows:

• Extracted the headers from the users' postings in OpenJPA Users' Mailing List.
• The extraction process was a breeze with the help of HTML Parser. The program found a list of 1059 independent poster headers.
• Fed the headers to Wordle to create a Word Cloud.

This is the Word Cloud after the first attempt

 

The original Word Cloud is here 

Of course, "problem" is often what makes a user to post in the forum. To focus further and because did not like "problem" being so prominent in the Word Cloud, removed the following words for the input list of headers: "OpenJPA", "entity", "mapping", "error", "using", "Re:"

The original Word Cloud is here

OpenJPA Enhancement Eclipse Builder

As I wrote about in a previous blog post, enhancement is HIGHLY recommended when developing an application that uses OpenJPA. The OpenJPA community has been vigilant about trying to cover every enhancement scenario, but after my last posting a reader brought up new somewhat uncovered situation. I say somewhat, because I'm sure the user could have enhanced using one of the existing methods, but it would not have been pretty. My understanding is that they are using Eclipse WTP to write and deploy their application, and orm.xml to map their Entities. We have the first two variants covered with an Eclipse plugin, but the third is something that we have not addressed. I did some research and found that creating a custom Ant builder may be the perfect fit. This builder will be executed by Eclipse after compilation has completed. Below I've documented the steps required to create an Ant builder that will enhance your Entities after each compile.

Please contact me with any questions/comments.

-Rick

For steps documented below, I have the following directory structure. Note, these steps must be followed for each project that has Entities that need to be enhanced.

/builder_project
enhance.xml <- the OpenJPA builder.... download here.
/bin <- Compile directory
/src <- Source directory
/jpa_lib <- OpenJPA binary and all jars from the lib dir of the binary download
commons-collections-3.2.jar
commons-lang-2.1.jar
commons-pool-1.3.jar
derby-10.2.2.0.jar
geronimo-jpa_2.0_spec-1.0-EA-SNAPSHOT.jar
geronimo-jta_1.1_spec-1.1.1.jar
openjpa-2.0.0-SNAPSHOT.jar
serp-1.13.1.jar
/lib <- other libs

1. After you add the enhance.xml file to your file system, make sure to refresh your Eclipse workspace so it knows about the newly added file. Make sure that the enhance.xml file is listed in the Navigator view.
   
2. Right click on the Eclipse project that you want to enhance and click on properties.
3. Click on the builders filter, and create a new Ant builder.
4. Name your builder, then click on "Browse Workspace" in the buildfile box. If you downloaded the enhance.xml file and refreshed your workspace, it should be listed there. If not, go back to step 1 and make sure that Eclipse detects your enhance.xml file.
   
5. In the "Base Directory" box, click on the variables button and select build_project. This should refer to the root of your project. In the directory structure above, it refers to "builder_project".
   
6. In the "Arguments" box you need to add the following properties -Dopenjpa.libs and -Dbuild.dir. -Dopenjpa.libs is the path to the OpenJPA libs, relative to the root of the project. -Dbuild.dir is the path to the build directory, relative to the root of the project. In the directory structure above, openjpa.libs should be set to jpa_lib and build.dir should be set to bin.
   
7. Click on the "Targets" tab along the top.
8. You need to set the enhance target to run as a part of "Manual Build" and "Auto Build".
   

Screenshots:

Files:
[1]http://filebin.ca/zbfbg/enhance.xml -- enhance.xml
[2]http://filebin.ca/aayeg/navigator.png -- Navigator pane
[3]http://filebin.ca/cznvmv/main.png -- Edit builder configuration - main
[4]http://filebin.ca/fqnadv/targets.png -- Edit builder configuration - targets

Jazz and Software Development

Jazz is said to be the fundamental rhythms of human life and man’s contemporary reassessment of his traditional values. The ancient beats and rhythms of Africa found a contemporary expression in the new world of New Orleans around early twentieth century. Often music historians spoke about undefinable quality of Jazz. Louis Armstrong -- one of the greatest Jazz musician ever - put is succinctly : "If you gotta ask, you’ll never know". The unscripted collaboration where every man plays his own tune but responds and respects others' -- is the core theme that gave rise to this uniquely American contribution to music.

It is intriguing that the new software collaboration platform built on Eclipse from contributions by pioneers such as Erich Gamma is named Jazz. Naming is very important. Naming captures aspiration. A software engineer as great as Erich Gamma can foresee the evolution of software development practices or at least where he wants it to go. Building software is playing Jazz -- not Orchestra.

Dynamic Fetch Planning

Now I have heard this before:

    "Hibernate can do X, can your thing do X?"

I mumble in a me-too manner. Because Hibernate, as the world knows, is 42 -- the answer to the eternal question. Also I have seen many postings in OpenJPA's own mailing list that can be paraphrased as: "We are migrating our Hibernate application to OpenJPA, but we could do X with Hibernate, OpenJPA does not seem to be able to do X". Or a slight variation on the theme: "This test fails with OpenJPA. But it passes with Hibernate".

As a contributor to OpenJPA, this implied second-class status irks me at times (I also wonder: why are they migrating away from Hibernate anyway?). So when the following email from an unknown sender reached my mailbox -- I smiled. The mail says:

"We are working in investment banking's IT sector. We have a requirement X that Hibernate can not support. Can OpenJPA do X?"

Firstly, I was happy to note that OpenJPA can do X very well.

Secondly, the proposal to include feature X to JPA 2.0 Specification was simply ignored.

Thirdly, investment banking's IT is the bellwether of enterprise technology trends and demands.

I must admit that I do not know Hibernate deep enough to ascertain that it can or can not do X -- I am just going by the sender's comments.

So what is this feature X? It is called Dynamic FetchPlan.

The conventional wisdom is to mix fetch plan with query. This wisdom is rooted in the history of SQL -- where projection and selection criteria appear together in an all-encompassing SQL statement (I have heard stories of a single SQL statement being 2000 lines long). JPA took the same route -- its query language JPQL introduced FETCH JOIN which lets you specify what instances (that are not directly part of the projected result) will be brought in the memory as a side-effect of query execution.

While OpenJPA supports fetch join as per JPA specification -- OpenJPA prefers to separate these two concerns, namely: i) what is selected and ii) what is fetched. Because fetch plan is independently specified, you can issue a simple EntityManager.find(Company.class, "acme.org") -- and as a result the persistent context may either  be populated with a single Company instance or with all the Departments and all the Employees of those Departments with their Addresses and Spouses' names. It all depends on what fetch plan is active while you invoked EntityManager.find(). In one end of the spectrum, you can use basic default fetch plan which fetches the fields of primitive types but no relations. On the other end, you can be as creative as you wish to specify a sub-graph of the complete closure starting from a root Company instance. And you can specify these fetch plans during design, use them and edit them at runtime.

But why should you care to carve out a sub-graph from a root entity? Why does OpenJPA use fetch plan as a pervasive notion throughout its internal design? Why even the question may be worth pondering?

My take on it will take a separate post -- more importantly Oscar is starting in TV....