Tuesday, 19 May 2015

Database Testing with H2

H2 is an in-memory database which can be really useful for running unit tests against.  It allows the JPA annotated classes (Entities) to be used to write to and read from a real database.

Schema Generation

Use the previous post on Schema Generation from JPA objects to make sure that a schema.sql file exists in the test resources directory.  This can be used to create the database tables for H2 to then use.

Pom Dependencies

There are a few dependencies that need to be included in the pom file if they aren't already.  These include JUnit, H2 and spring, as well as hibernate entity manager.

    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.12</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>com.h2database</groupId>
        <artifactId>h2</artifactId>
        <version>1.3.174</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-test</artifactId>
        <version>${spring.version}</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-context</artifactId>
        <version>${spring.version}</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-orm</artifactId>
        <version>${spring.version}</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.hibernate</groupId>
        <artifactId>hibernate-entitymanager</artifactId>
        <version>${hibernate.version}</version>
        <scope>test</scope>
    </dependency>

Spring Context for H2

This spring context (domain-test-context.xml) defines the use of H2 as the data source and initialises the database with a createSchema.sql file and the schema.sql file generated above by the hibernate4 maven plugin.

<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:jdbc="http://www.springframework.org/schema/jdbc"
    xmlns:tx="http://www.springframework.org/schema/tx"
    xsi:schemaLocation="  
        http://www.springframework.org/schema/beans       
        http://www.springframework.org/schema/beans/spring-beans-4.0.xsd  
        http://www.springframework.org/schema/tx       
        http://www.springframework.org/schema/tx/spring-tx-4.0.xsd  
        http://www.springframework.org/schema/jdbc   
        http://www.springframework.org/schema/jdbc/spring-jdbc-4.0.xsd" >

    
    <jdbc:embedded-database id="dataSource" type="H2" />
    
    <jdbc:initialize-database data-source="dataSource" ignore-failures="ALL">
        <jdbc:script location="createSchema.sql" />
        <jdbc:script location="schema.sql" />
        <!-- We could put other initialising data stuff in here -->
    </jdbc:initialize-database>
    
    <!-- This manages the entities and interactions with the database -->
    <bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
        <property name="packagesToScan" value="com.my.classes.impl" />
        <property name="dataSource" ref="wiDataSource" />
        <property name="jpaVendorAdapter">
            <bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
                <property name="showSql" value="false" />
                <property name="databasePlatform" value="org.hibernate.dialect.H2Dialect" />
                <property name="generateDdl" value="false" />
            </bean>
        </property>
        <property name="jpaProperties">
            <props>
                <!-- General settings -->
                <prop key="hibernate.archive.autodetection">class, hbm</prop>
                <prop key="hibernate.current_session_context_class">jta</prop>
                <prop key="hibernate.show_sql">true</prop>
                <prop key="hibernate.format_sql">true</prop>
            </props>
        </property>
    </bean>
    
    <tx:annotation-driven transaction-manager="transactionManager" proxy-target-class="true" />
    
    <bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
        <property name="entityManagerFactory" ref="entityManagerFactory" />
    </bean>
    
    <bean class="org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor" />
       
    <bean id="myRepository" class="com.my.classes.repo.MyRepository" />
</beans>

This context creates the H2 database and wires the MyRepository ready for testing.

The createSchema.sql file contains the line as the schema.sql generated won't generate a database schema itself only the tables / columns which are part of that schema.

create schema MY_SCHEMA;

Test Class

This is the sample test class.  It uses the Spring Runner with the text context above.  It is ApplicationContextAware so that we can load the repository (@Autowire could be used instead) and trust spring to use the PersistenceAnnotationBeanPostProcessor to inject the EntityManager.

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = {"classpath:domain-test-context.xml"})
public class WorkInstructionRepositoryTest implements ApplicationContextAware
{
    /**
     * This is the application context that we can load beans from.
     */
    private ApplicationContext applicationContext;

    /**
     * JPA Entity Manager.
     */
    @PersistenceContext
    private EntityManager entityManager;

    /**
     * The class under test.
     */
    private MyRepository repo;

    /**
     * {@inheritDoc}
     */
    @Override
    public void setApplicationContext(final ApplicationContext applicationContext) throws BeansException
    {
        this.applicationContext = applicationContext;
    }

    /**
     * Before each test get the repository.
     */
    @Before
    public void before()
    {
        this.repo = (MyRepository) applicationContext.getBean("myRepository");
    }

    /**
     * Test creating and loading some data.
     */
    @Test
    @Transactional
    @Rollback(true)
    public void testCreateAndReadData()
    {
        // Arrange
        final SomeData data = new SomeData();
        data.setActive(true);
        data.setText("Some Data Text");

        // Act
        repo.createData(data);
        final List<DataInterface> loadedDataObjs = repo.getAllData();

        // Assert
        final DataInterface loadedData = loadedDataObjs.get(0);
        assertEquals(data.isActive(), loadedData.isActive());
        assertEquals(data.getText(), loadedData.getText());
    }
}

JPA Schema Generator

This is a sample of using the de.juplo plugin for generating a database schema from annotated JPA Entity classes.

The Config


 <plugin>
    <groupId>de.juplo</groupId>
    <artifactId>hibernate4-maven-plugin</artifactId>
    <version>1.1.0</version>
    <executions>
        <execution>
            <id>package</id>
            <phase>prepare-package</phase>
            <goals>
                <goal>export</goal>
            </goals>
            <configuration>
                 <hibernateDialect>org.hibernate.dialect.MySQL5Dialect</hibernateDialect>
            </configuration>
        </execution>
        <execution>
            <id>test</id>
            <phase>process-test-resources</phase>
            <goals>
                <goal>export</goal>
            </goals>
            <configuration>
                <outputFile>${project.build.testOutputDirectory}/schema.sql</outputFile>
                <hibernateDialect>org.hibernate.dialect.H2Dialect</hibernateDialect>
            </configuration>
        </execution>
    </executions>
    <configuration>
        <target>SCRIPT</target>
    </configuration>
 </plugin>

Note that the normal configuration doesn't need an outputFile because it is generated in the target directory by default.  This example uses two executions, the first for the standard target build and the second for a test build for use with H2 for unit testing.
The <target>SCRIPT</target> part means that a sql script is created rather than trying to execute this directly on a database.

For more information see http://juplo.de/hibernate4-maven-plugin/

Wednesday, 11 March 2015

JavaFX Filter ComboBox

Quite a lot of posts exist on the web about how hard it is to do a filter / filtered / filtering combo box in JavaFX.  Using one of these as a basis here is an example of what to do.  The key part is creating a StringConverter so that the strings that are displayed in the ComboBox can be translated to and from the underlying objects.

FilterComboBox

This class is a extension of the standard JavaFX ComboBox. The list of initialItems needs to be populated either through the constructor or by using the setInitialItems() method so that there is a definitive list to go back to when necessary.

import java.util.ArrayList;

/**
 * A control which provides a filtered combo box.  As the user
 * enters values into the combo editor the list is filtered automatically.
 *
 * @param <T> the object type that is held by this FilteredComboBox
 */
public class FilterComboBox<T extends Object> extends ComboBox<T>
{
    /**
     * The default / initial list that is in the combo when nothing
     * is entered in the editor.
     */
    private Collection<T> initialList = new ArrayList<>();

    /**
     * Check type.  True if this is startsWith, false if it is contains.
     */
    private final boolean startsWithCheck;

    /**
     * Constructs a new FilterComboBox with the given parameters.
     *
     * @param startsWithCheck true if this is a 'startsWith' check false if it is 'contains' check
     */
    public FilterComboBox(final boolean startsWithCheck)
    {
        super();
        this.startsWithCheck = startsWithCheck;

        super.setEditable(true);

        this.configAutoFilterListener();
    }

    /**
     * Constructs a new FilterComboBox with the given parameters.
     *
     * @param items The initial items
     * @param startsWithCheck true if this is a 'startsWith' check false if it is 'contains' check
     */
    public FilterComboBox(final ObservableList<T> items, final boolean startsWithCheck)
    {
        super(items);
        this.startsWithCheck = startsWithCheck;
        super.setEditable(true);
        initialList = items;

        this.configAutoFilterListener();
    }

    /**
     * Set the initial list of items into this combo box.
     *
     * @param initial The initial list
     */
    public void setInitialItems(final Collection<T> initial)
    {
        super.getItems().clear();
        super.getItems().addAll(initial);
        this.initialList = initial;
    }

    /**
     * Set up the auto filter on the combo.
     */
    private void configAutoFilterListener()
    {
        this.getEditor().textProperty().addListener(new ChangeListener<String>()
        {
            @Override
            public void changed(final ObservableValue<? extends String> observable, final String oldValue, final String newValue)
            {
                final T selected = getSelectionModel().getSelectedItem();
                if (selected == null
                        || !getConverter().toString(selected).equals(getEditor().getText()))
                {
                    filterItems(newValue);

                    if (getItems().size() == 1)
                    {
                        setUserInputToOnlyOption();
                        hide();
                    }
                    else if (!getItems().isEmpty())
                    {
                        show();
                    }
                }
            }
        });
    }

    /**
     * Method to filter the items and update the combo.
     *
     * @param filter The filter string to use.
     */
    private void filterItems(final String filter)
    {
        final ObservableList<T> filteredList = FXCollections.observableArrayList();
        for (T item : initialList)
        {
            if (startsWithCheck && getConverter().toString(item).toLowerCase().startsWith(filter.toLowerCase()))
            {
                filteredList.add(item);
            }
            else if (!startsWithCheck && getConverter().toString(item).toLowerCase().contains(filter.toLowerCase()))
            {
                filteredList.add(item);
            }
        }

        setItems(filteredList);
    }

    /**
     * If there is only one item left in the combo then we assume this is correct.
     * Put the item into the editor but select the end of the string that the user
     * hasn't actually entered.
     */
    private void setUserInputToOnlyOption()
    {
        final String onlyOption = getConverter().toString(getItems().get(0));
        final String currentText = getEditor().getText();
        if (onlyOption.length() > currentText.length())
        {
            getEditor().setText(onlyOption);
            Platform.runLater(new Runnable()
            {
                @Override
                public void run()
                {
                    getEditor().selectAll();
                }
            });
        }
    }
}

StringConverter

The key to this working correctly is a StringConverter object which allows JavaFX to convert properly from the Object in the ComboBox to the String which is displayed in the ComboBox.  The StringConverter is a standard JavaFX object which has a toString() and fromString() method.  The easiest way to get this working is to construct a StringConverter and provide the same list of items to it that is provided to the ComboBox.


public class MyObjectStringConverter extends StringConverter<MyObject> 
{
    /** The list of objects to do the conversions with. */
    private List<MyObject> myObjList;

    /**
     * Construct this object with the list for converting.
     *
     * @param items The items list
     *
     */
    public MyStringConverter(List<MyObject> items)
    {
        this.myObjList = items;
    }

    @Override
    public String toString(final MyObject myObj)
    {
        if (myObj != null)
        {
            return myObj.getName();
        }
        return null;
    }

    @Override
    public MyObject fromString(final String item)
    {
        for (MyObject myObj : myObjList)
        {
            if (myObj.getName().equals(item))
            {
                return myObj;
            }
        }
        return null;
    }
}

Usage

The usage of this combo is extremely easy.  Other than the standard ComboBox options in JavaFX the only things really necessary are making sure the initialList is set through the constructor or directly and making sure that the converter is used.

    final List<MyObject> myObjs = new ArrayList<>();
    ...

    final MyObjectStringConverter converter = new MyObjectStringConverter(myObjs);

    final FilterComboBox comboBox = new FilterComboBox(true);
    comboBox.setInitialItems(myObjs);
    comboBox.setConverter(converter);

Friday, 22 August 2014

Profiling & jvisualvm

JVisualVm is a really good profiling tool for use with Java.  It is included in the JDK.

To use it on a local running instance then just run it.  All the VMs will appear in the application menu on the left hand side.

Remote
JVisualVM can also be used remotely.  On the machine with the VM to profile, create a policy file which contains the following

// create a policy file called all.policy
grant codebase "file:${java.home}/../lib/tools.jar" {
   permission java.security.AllPermission;

};

Then create a RMI instance on a particular local port (1099 is standard but any port can be used such as 2020)

// Start the rmiregistry

rmiregistry 1099&

And now use jstatd to populate the registry.

// Start jstatd with the policy file above and point it to the rmiregistry

jstatd -J-Djava.security.policy=all.policy -p 1099

Now start jvisualvm on a local machine and connect to port 1099 on the remote machine to view the details.

Within jvisualvm right click on 'Remote' and select 'Add remote host'.  In the hostname add the IP address of the remote machine and in 'Advanced' the port can be change if 1099 isn't used for the rmiregistry or jstatd.

It isn't possible to do everything with a remote connection that you can do with a local connection. For this reason it can be useful to run jvisualvm locally but use X11 forwarding to put the actual application on a remote machine.

Troubleshooting

Sometimes there are connection issues when trying to connect to a remove jstatd.  So check the following

Logs
Look at the jvisualvm logs which can be found in ~/.visualvm/<version>/var/log

Headless
Although jvisualvm can be run with X11 forwarding so it can be a headless environment it has dependencies on some X11 libraries so needs to have a desktop manager installed even if it isn't running. (libXext.so.6)

Firewall
Make sure that any firewall isn't blocking the RMI ports.

RMI address
Sometimes the RMI address needs to be set so try adding

    -J-Djava.rmi.server.hostname=<ip address>

IP version
It is also possible to try to force IPv4 to see if that helps,

    -J-Djava.net.preferIPv4Stack=true 


Other Tools

There are a number of other tools which are useful to ascertaining if there is a memory leak in the heap.  A heap dump can be generated using the jmap command which is part of the jdk

    ./jmap -dump:format=b,file=<filepath> <pid>

This generates a hprof format.  There is a free tool called the Memory Analyser Tool (MAT) which is based on the eclipse platform and can be used to analyse the heap file.

    http://www.eclipse.org/mat/

Wednesday, 13 August 2014

@Transactional

When connecting to a database a transaction is often required.  There are a number of ways of doing this but the most common is to use Spring transactions.

pom.xml

This dependency is needed to allow transactions

<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-tx</artifactId>
<scope>compile</scope>
</dependency>


Spring Context

In the spring configuration an entity manager is necessary (see Entity Manager blog entry)

<tx:annotation-driven transaction-manager="transactionManager" proxy-target-class="true" />

<!-- Define a transaction manager so that the @TransactionConfiguration and @Transactional can be used -->
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>

Proxy Class

The above combination will allow spring to create proxy classes which create a transaction before passing into the @Transactional method.  This means that because it is a proxy @Transactional doesn't work for any 'internal' call within the your class.  Eg even if a method is public if it is called from within the same class it won't pass through the proxy and therefore won't have a transaction.
If this type of behaviour is required then the transactions can be wired using AOP and the transaction code is created at compile time rather than a runtime proxy.

Exception Handling

Normal exceptions can be caught using the normal try - catch but if the database isn't there at all then there is an exception generated in the proxy before the method code is called.  The exception is a 
    org.springframework.transaction.CannotCreateTransactionException

To catch the exception within the same method as the actual database errors the transaction can be created a different way.

Programmatic Transactions

A programmatic transaction can be created using the TransactionTemplate in which case the TransactionException (CannotCreateTransactionException) can be caught in the same place.

    try
    {
        transactionTemplate.execute(new TransactionCallbackWithoutResult()
        {
            @Override
            protected void doInTransactionWithoutResult(final TransactionStatus status)
            {
                 // Do something to the database here
            }
        });
    }
    catch (PersistenceException | TransactionException e)
    {
        LOG.error("Database Error", e);
    }

There is also a TransactionCallback() which returns the object from the database whereas the one above doesn't do that.

The transactionTemplate can be created in spring configuration,

    <property name="transactionTemplate">
        <bean class="org.springframework.transaction.support.TransactionTemplate">
            <constructor-arg ref="transactionManager" />
</bean>
    </property>






Friday, 20 June 2014

Spring Web App Template

I don't create new web apps that often and therefore can easily forget the necessary bits!  This is my memory jogger for how do to these.

Folders
Notes: In eclipse create a new java project with the correct maven folder structure.  This is the sample folder structure

    project_home/src
    project_home/src/main
    project_home/src/main/java
    project_home/src/main/resources
    project_home/src/main/webapp
    project_home/src/main/webapp/WEB-INF
    project_home/src/main/webapp/WEB-INF/jsp
    project_home/src/test
    project_home/src/test/java
    project_home/src/test/resources

pom.xml
Location: project_home/pom.xml
Notes: The basic maven pom which has the correct values for the spring classes although the spring versions can be updated. 

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>

    <groupId>com.me.app</groupId>
    <artifactId>me-web</artifactId>
    <packaging>war</packaging>
    <version>0.0.1-SNAPSHOT</version>
    <name>Sample Web App</name>
    <description />
    <properties>
        <spring.version>3.1.2.RELEASE</spring.version>
    </properties>

    <dependencies>

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
            <version>${spring.version}</version>
        </dependency>

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-webmvc</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <!-- test -->
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-test</artifactId>
            <version>${spring.version}</version>
            <scope>test</scope>
        </dependency>

    </dependencies>

</project>

Eclipse
Notes: After putting this pom in place convert to a maven project.  In eclipse right click on the project and go to Configure - Convert to Maven Project.  This should create a folder called Maven Dependencies which contains all the libraries defined by the pom.
Right click on the project and go to properties.  
 - Select 'Project Facets' and make sure it is a faceted form with Dynamic Web Module, Java and JavaScript are all checked.
 - Click on Deployment Assembly and make sure that Source /src/main/webapp is mapped to Deploy Path / and also that Source: Maven Dependencies are mapped to Deploy Path /WEB-INF/lib.
 - Click on Web Project Settings and check the context-root value
 - Click on Java Build Path and add main/java, main/resources, test/java and test/resources as source folders.

web.xml
Location: project_home/src/main/webapp/WEB-INF/web.xml
Notes: This is the file that is read by the application server to determine details about the servlet it needs to run.  It contains the main servlet class and mappings of which requests get sent to this servlet.  In this case the servlet is the spring MessageDispatchServlet as this is a spring app.
   
 <web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee   
          http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"  
    version="2.5">
  <display-name>Web App</display-name>
  
  <servlet>  
        <servlet-name>webapp</servlet-name>  
        <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>  
        <load-on-startup>1</load-on-startup>  
    </servlet>  
  
    <servlet-mapping>  
        <servlet-name>webapp</servlet-name>  
        <url-pattern>/app/*</url-pattern>  
    </servlet-mapping>  
  
</web-app>


servlet.xml
Location: project_home/src/mainwebapp/WEB-INF/servlet.xml
Notes: The servlet xml is the file that is used by the spring MessageDispatcherServlet and is the root of the spring configuration for the web application.

<beans xmlns="http://www.springframework.org/schema/beans"  
    xmlns:context="http://www.springframework.org/schema/context"  
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
    xsi:schemaLocation="  
        http://www.springframework.org/schema/beans       
        http://www.springframework.org/schema/beans/spring-beans-3.0.xsd  
        http://www.springframework.org/schema/context   
        http://www.springframework.org/schema/context/spring-context-3.0.xsd">  
   
    <context:component-scan base-package="com.me.app.web" />  
   
    <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">  
        <property name="prefix">  
            <value>/WEB-INF/jsp/</value>  
        </property>  
        <property name="suffix">  
            <value>.jsp</value>  
        </property>  
    </bean>  
   
</beans> 


Controller
Location: project_home/src/main/java/com/me/app/web/controller/Greeting.java
Notes: This is a sample controller that maps to a jsp.

package com.me.app.web.controller;

import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.servlet.ModelAndView;

/**
 * Sample controller.
 */
@Controller
@RequestMapping("/greeting")
public class Greeting
{
    /**
     * Method to say hello.
     */
    @RequestMapping("/hello")
    public ModelAndView hello()
    {
        final ModelAndView mav = new ModelAndView("hello");
     return mav;
    }
}

jsp
Location: project_home/src/webapp/WEB-INF/jsp/hello.jsp
Notes: This is a very simple jsp file that is used to just smoke test that the application is running.

<html>  
  <head>
    <title>Sample Web App</title>
  </head>  
  <body>
    <center>  
      <h2>Sample Web App</h2>  
      <h4>Hello</h4>  
    </center>  
  </body>  
</html>  

Thursday, 19 June 2014

Dynamically Updating JavaFX Table

There are many tricks on the web for updating tables when a value changes and the table doesn't update such as adding and removing columns to force the table to refresh.  I think there may be a refresh table option in Java 8 now.  However, getting the data structure correct makes a big difference.

Using Properties
Use properties in the backing object so that the table can bind to them and get the updates.  In the example below the name is immutable so doesn't need a property but the count is changeable so use a property value and importantly include the countProperty() method.


public class MyTableData
{
    /**
     * A value that doesn't change so just stored as a string.
     */
    private final String name;
        
    /**
     * A changeable value so stored as a property.
     */
    private final SimpleIntegerProperty count;
    
    /**
     * Constructs a new MyTableData with the given parameters.
     *
     * @param name The name
     * @param count The count
     */
    public MyTableData (final String name, final Integer count)
    {
        this.name= name;
        this.count = new SimpleIntegerProperty(count);
    }

    /**
     * Gets the namevalue.
     *
     * @return the name
     */
    public String getName()
    {
        return name;
    }

    /**
     * Gets the count value.
     *
     * @return the count
     */
    public int getCount()
    {
        return count.get();
    }

    /**
     * Set the count value.
     *     * @param count The new value to set
     */
    public void setCount(final int count)
    {
        this.count.set(count);

    }

    /**
     * The count property.
     *
     * @return The count property
     */
    public IntegerProperty countProperty() 
    {
        return count;
    }
}

The Table
In the table create the columns as normal,

        // Count Column
        final TableColumn<MyTableData, Number> countCol = new TableColumn<MyTableData, Number>("Count");
        countCol.setCellValueFactory(new PropertyValueFactory<MyTableData, Number>("count"));