Sunday, February 22, 2009

Dynamic Fetch Planning

Now I have heard this before:

    "Hibernate can do X, can your thing do X?"

I mumble in a me-too manner. Because Hibernate, as the world knows, is 42 -- the answer to the eternal question. Also I have seen many postings in OpenJPA's own mailing list that can be paraphrased as: "We are migrating our Hibernate application to OpenJPA, but we could do X with Hibernate, OpenJPA does not seem to be able to do X". Or a slight variation on the theme: "This test fails with OpenJPA. But it passes with Hibernate".

As a contributor to OpenJPA, this implied second-class status irks me at times (I also wonder: why are they migrating away from Hibernate anyway?). So when the following email from an unknown sender reached my mailbox -- I smiled. The mail says:

"We are working in investment banking's IT sector. We have a requirement X that Hibernate can not support. Can OpenJPA do X?"

Firstly, I was happy to note that OpenJPA can do X very well.

Secondly, the proposal to include feature X to JPA 2.0 Specification was simply ignored.

Thirdly, investment banking's IT is the bellwether of enterprise technology trends and demands.

I must admit that I do not know Hibernate deep enough to ascertain that it can or can not do X -- I am just going by the sender's comments.

So what is this feature X? It is called Dynamic FetchPlan.

The conventional wisdom is to mix fetch plan with query. This wisdom is rooted in the history of SQL -- where projection and selection criteria appear together in an all-encompassing SQL statement (I have heard stories of a single SQL statement being 2000 lines long). JPA took the same route -- its query language JPQL introduced FETCH JOIN which lets you specify what instances (that are not directly part of the projected result) will be brought in the memory as a side-effect of query execution.

While OpenJPA supports fetch join as per JPA specification -- OpenJPA prefers to separate these two concerns, namely: i) what is selected and ii) what is fetched. Because fetch plan is independently specified, you can issue a simple EntityManager.find(Company.class, "") -- and as a result the persistent context may either  be populated with a single Company instance or with all the Departments and all the Employees of those Departments with their Addresses and Spouses' names. It all depends on what fetch plan is active while you invoked EntityManager.find(). In one end of the spectrum, you can use basic default fetch plan which fetches the fields of primitive types but no relations. On the other end, you can be as creative as you wish to specify a sub-graph of the complete closure starting from a root Company instance. And you can specify these fetch plans during design, use them and edit them at runtime.

But why should you care to carve out a sub-graph from a root entity? Why does OpenJPA use fetch plan as a pervasive notion throughout its internal design? Why even the question may be worth pondering?

My take on it will take a separate post -- more importantly Oscar is starting in TV....


nits said...

Fetch Groups was the thing which got me interested in looking at Open JPA as an alternative to Hibernate. As far as I remember, the request for similar feature has been raise a couple of times on hibernate forum and turned down. I am really disappointed that it is not a part of JPA 2.0 spec. In most of my project, marking the associations as lazy or eager has become an architectural decision, Fetch Group now makes it very simple and use case specific instead of design.

Kevin Sutter said...

Thanks for the comments. We like our Fetch Plan and Fetch Group support as well... The idea of standardizing this support continues to come up during the JPA expert group discussions, but it just hasn't made the cut yet. Thanks for the feedback.

Seb said...

I think in the book POJOs in Action, the case is made pretty clear the need for fetch groups when in the perspective of a well design application.

It's less clear from a laboratory perspective, from people writing small unit test to show their framework features.

In the J2EE Patterns, Transfer Object Assembler is responsible to know what need to be available in the object tree. The assembler is far from the queries, many queries.

In complex application a subset of queries will be involve by many different assemblers having different needs. They is not much other alternatives than DECLARING what need to be fetched in the assembler before calling indirectly an undefined number of queries hidden deep in the layers.

aj said...

I have a similar scenario. I need my entity manager operations like find and merge act differently. On my merge i have to merge the null, so i have the confifuration all(detachedState=true). So this made all my entities along with its childs to be loaded to memory. But on my selects( em.find()), i dont want all my childs to be retrived, just the root is sufficient. Tried with fecthplans, pushFechPlan, even with custom fetch groups. Fetch plans, does nt seem to be working. I could see the setMaxDepth, when dynamically change the fecthplan, has the value as 0 for em.find() and em.merge has infinites as expected(-1). But still when it fires SQL, i can see its loading all entities irrespective of setting the max depth to root.Any help on this will be appreciated.

Kevin Sutter said...

Hi AJ,
What you seem to be describing should be doable with OpenJPA's fetch plans and/or fetch groups. You might want to reference the latest documentation to see if you can figure out what's different:

One thing to watch out for is the use of the "default" fetch group versus a user-defined fetch group. It may be that you are accidentally falling back to the "default" fetch group.

If none of this helps, then I would suggest posting to the OpenJPA users mailing list. Other users and/or developers may have some additional insights for you. And, if a problem is discovered, then a bug report can be opened.

Good luck,

for IT the said...

Enjoyed your approach to explaining how it works, hope to see more blog posts from you. thank you!

Hibernate Online Training | Java Online Training

Hibernate Training in Chennai Java Training in Chennai