NetBeans & JPA with Multiple Persistence.xml Files

If you are developing a large, desktop application which needs to keep a persistent state one of a complex data structure, where perhaps you have also to perform queries, the use of an embedded relational database with an ORM is a good solution. My experience with blueMarine has been quite positive so far with Derby as the embedded database and Hibernate as ORM. Since a few months I'm slowly replacing all the Hibernate-specific code with JPA, to take advantage of annotations and to have more reusable code (for instance, it could be used in other projects with different ORMs such as TopLink, OpenJPA, etc).

As a side note, this code at the moment doesn't use any of the NetBeans RCP specific APIs, so it can be used if you have to deal with multiple JPA configuration files even in other environments.

Now, blueMarine is highly modular; I like modularity in projects for a number of (obvious) reasons and I have more than one hundred modules in it, developed around the NetBeans RCP APIs. Some of these modules contain their own persistence objects (entities) and they should have their own, independent ORM mappings. With Hibernate, this was not a problem: each module had a set of *.hbm.xml files containing the class-table mappings and you could manually load them one by one with this approach:

import org.hibernate.Session;
import org.hibernate.SessionFactory;
import org.hibernate.cfg.Configuration;

...

SessionFactory sessionFactory;

...

Configuration configuration = new Configuration();
InputStream is = ... get resource input stream for the *.hbm.xml file
configuration.addInputStream(is);
sessionFactory = configuration.buildSessionFactory()

 

The important point is to re-create the sessionFactory every time you modify the configuration, which could be a bit annoying, but in the end it happens only a few times during the initialization of the application (or the first time you dynamically activate a module with a persistent entity inside).

With JPA things change. The ORM mappings are loaded by default from annotations, so there's no need for the *.hbm.xml files, and this is a good simplification; but there is no API for programmatically register persistent entities, that must be declared in a specific persistence.xml file such in this example:

<?xml version="1.0" encoding="UTF-8"?>
<persistence version="1.0" xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd">
  <persistence-unit name="BLUEMARINE_PU" transaction-type="RESOURCE_LOCAL">
    <class>it.tidalwave.catalog.persistence.AttributePB</class>
    <class>it.tidalwave.catalog.persistence.CategoryPB</class>
    <class>it.tidalwave.catalog.persistence.TagBindingPB</class>
    <class>it.tidalwave.catalog.persistence.TagPB</class>
...
  </persistence-unit>
</persistence>  

 

If you are used to JPA and J2EE this might sound as a little surprise: in J2EE environments you don't need to explicitly list all the entities, they are dynamically discovered by classpath inspection; outside J2EE you need to explicitly list them (I don't know the reason of this difference, perhaps the point is that classpath inspection takes a bit of time and you can afford to pay it when you re-deploy a J2EE application).

As a second note, JPA indeed allows multiple persistence.xml files, but only if they are related to different persistence units (PUs) - a PU is the container of all the configuration, caches and such for a group of persistent objects, and I don't want to have multiple ones, but a single one for the whole application ("BLUEMARINE_PU").

So, what to do? Thanks to the suggestions of Andrei Badea, I've found a solution, with just two minor issues to fix. The steps to perform are:

1. Scan through the classpath and find all the persistence.xml files;
2. Manipulate them by means of XML APIs and merge into a single file;
3. Feed the resulting file to JPA

The complete code is in the ClassLoaderProxy class, that is available through subversion here (use -r 5502, which is the current while I'm typing, as it will be surely refactored in future).

First, our class extends a ClassLoader

public class ClassLoaderProxy extends ClassLoader 
  {
    ...
    public ClassLoaderProxy (final ClassLoader parent)
      {
        super(parent);
      }
    ...
  }

The first method we deal with is able to scan the classpath for the JPA configuration files:

    private static final String PERSISTENCE_XML = "META-INF/persistence.xml";

    ...

    private Collection<URL> findPersistenceXMLs (final Filter filter)
      throws IOException
      {
        final Collection<URL> result = new ArrayList<URL>();
        
        for (final Enumeration<URL> e = super.getResources(PERSISTENCE_XML); e.hasMoreElements(); )
          {
            final URL url = e.nextElement();
            
            if (filter.filter(url))
              {
                result.add(url);
              }
          }
        
        return result;
      }

The filter parameter is needed because my approach is to have a "master" persistence.xml file, with all the generic configuration, and several "secondary" files that just list the persistent entities inside each module:

    private static final String MASTER_URL_SUFFIX = "it-tidalwave-bluemarine-persistence.jar!/" + PERSISTENCE_XML;
    
    enum Filter
      {
        MASTER
          {
            public boolean filter (URL url)
              {
                return url.toExternalForm().endsWith(MASTER_URL_SUFFIX);
              }
          },
          
        OTHERS
          {
            public boolean filter (URL url)
              {
                return !url.toExternalForm().endsWith(MASTER_URL_SUFFIX);
              }
          };
        
        public abstract boolean filter (URL url);
      }    

The MASTER_URL_SUFFIX, as you can see, contains the jar name of the module where the master persistence.xml file is.

Now, the code that gets all the pieces and merge them. Basically, it extracts all the "classes" declarations from the secondary files and inserts them in the master file. I'm not giving details on the implementation, as it is just a JAXP / XPATH exercise:

    private static final DocumentBuilderFactory DOC_BUILDER_FACTORY = DocumentBuilderFactory.newInstance();
        
    private static final XPathFactory XPATH_FACTORY = XPathFactory.newInstance();
        
    private static final XPath XPATH = XPATH_FACTORY.newXPath();
        
    private static final XPathExpression XPATH_ENTITY_PU_NODE;
    private static final XPathExpression XPATH_ENTITY_CLASS_TEXT;

    ...

    static
      {
        try
          {
            XPATH_ENTITY_PU_NODE = XPATH.compile("//persistence/persistence-unit");
            XPATH_ENTITY_CLASS_TEXT = XPATH.compile("//persistence/persistence-unit/class/text()");
          }
        catch (XPathExpressionException e) 
          {
            throw new ExceptionInInitializerError(e);
          }
      }

    ...

    private String scanPersistenceXML()
      throws IOException, 
             ParserConfigurationException, 
             SAXException, 
             XPathExpressionException, 
             TransformerConfigurationException, 
             TransformerException
      {
        logger.info("scanPersistenceXML()");
        final DocumentBuilder builder = DOC_BUILDER_FACTORY.newDocumentBuilder();
        DOC_BUILDER_FACTORY.setNamespaceAware(true); 
        
        final URL masterURL = findPersistenceXMLs(Filter.MASTER).iterator().next();
        logger.fine(String.format(">>>> master persistence.xml: %s", masterURL));
        final Document masterDocument = builder.parse(masterURL.toExternalForm());
        final Node puNode = (Node)XPATH_ENTITY_PU_NODE.evaluate(masterDocument, XPathConstants.NODE);
              
        for (final URL url : findPersistenceXMLs(Filter.OTHERS))
          {
            logger.info(String.format(">>>> other persistence.xml: %s", url));
            final Document document = builder.parse(url.toExternalForm());
            final NodeList nodes = (NodeList)XPATH_ENTITY_CLASS_TEXT.evaluate(document, XPathConstants.NODESET);
              
            for (int i = 0; i < nodes.getLength(); i++) 
              {
                final String entityClassName = nodes.item(i).getNodeValue();
                logger.info(String.format(">>>>>>>> entity class: %s", entityClassName));
                
                if (i == 0)
                  {
                    puNode.appendChild(masterDocument.createTextNode("\n"));
                    puNode.appendChild(masterDocument.createComment(" from " + url.toExternalForm().replaceAll(".*/cluster/modules/", "") + " "));
                    puNode.appendChild(masterDocument.createTextNode("\n"));
                  }
                
                final Node child = masterDocument.createElement("class");
                child.appendChild(masterDocument.createTextNode(entityClassName));
                puNode.appendChild(child);
                puNode.appendChild(masterDocument.createTextNode("\n"));
              }
          }
        
        return toString(masterDocument);
      }

Now, where should we trigger this processing? In the getResource() method of the ClassLoader, which is precisely the one called by JPA to retrieve the persistence.xml file. This method must return the URL of the requested resource and the idea is that we will not return the master file URL, rather the URL of our synthetised file.

private static final String PERSISTENCE_XML = "META-INF/persistence.xml";
    
    private URL persistenceXMLurl;
    
    ...

    @Override
    public Enumeration<URL> getResources (final String name) 
      throws IOException 
      {
        if (PERSISTENCE_XML.equals(name))
          {
            if (persistenceXMLurl == null)
              {
                try 
                  {
                    final String persistenceXml = scanPersistenceXML();
                    logger.fine("persistence.xml " + persistenceXml);
                    
                    // The base directory must be empty since Hibernate will scan it searching for classes.
                    final File file = new File(System.getProperty("java.io.tmpdir") + "/blueMarinePU/" + PERSISTENCE_XML);
                    file.getParentFile().mkdirs();
                    final PrintWriter pw = new PrintWriter(new FileWriter(file));
                    pw.print(persistenceXml);
                    pw.close();
                    persistenceXMLurl = new URL("file://" + file.getAbsolutePath());
                    logger.info("URL: " + persistenceXMLurl);
                  } 
                catch (ParserConfigurationException e) 
                  {
                    throw new IOException(e.toString());
                  } 
                catch (SAXException e) 
                  {
                    throw new IOException(e.toString());
                  } 
                catch (XPathExpressionException e) 
                  {
                    throw new IOException(e.toString());
                  } 
                catch (TransformerConfigurationException e)
                  {
                    throw new IOException(e.toString());
                  } 
                catch (TransformerException e) 
                  {
                    throw new IOException(e.toString());
                  } 
              }
            
            return new Enumeration<URL>() 
              {
                URL url = persistenceXMLurl;
                
                public boolean hasMoreElements() 
                  {
                    return url != null;
                  }

                public URL nextElement() 
                  {
                    final URL url2 = url;
                    url = null;
                    return url2;
                  }
              };
          }
        
        return super.getResources(name);
      }

 

Here you can see the first rough corner. Since I must return an URL (that will be later opened by JPA by opening a connection to it), I must create a physical file to refer it (indeed NetBeans RCP allows to create a "virtual" filesystem which is mapped into memory and it has its own URLs - I attempted to use it, but I've been told that those URLs can't be connected to, so this approach won't work. I think it can be fixed, but it's a matter of a future post).

Well, we're at the end of the job. Now we just have to force JPA to use our ClassLoader, instead of the default one. In blueMarine, I have a specific method of a specific service that is used to access JPA. In this method, I'm just overriding the "system ClassLoader" across the call to JPA, as in the following code:

    public synchronized EntityManagerFactory getEntityManagerFactory()
      {
        if (entityManagerFactory == null)
          {
            final Properties properties = configuration.getProperties();
            final Thread currentThread = Thread.currentThread();
            final ClassLoader saveClassLoader = currentThread.getContextClassLoader();
            currentThread.setContextClassLoader(new ClassLoaderProxy(saveClassLoader));
            entityManagerFactory = javax.persistence.Persistence.createEntityManagerFactory("BLUEMARINE_PU", configuration.getProperties()); 
            currentThread.setContextClassLoader(saveClassLoader);
          }
        
        return entityManagerFactory;
      }

This is the second rough corner: while it fits my needs, a better solution would be to try to replace the system class loader globally; in this way, you could call JPA without any specific care. At the moment I don't know whether it's possible to replace the system ClassLoader in NetBeans RCP - matter for another post!