Profiling & jvisualvm

JVisualVm is a really good profiling tool for use with Java.  It is included in the JDK.

To use it on a local running instance then just run it.  All the VMs will appear in the application menu on the left hand side.

Remote
JVisualVM can also be used remotely.  On the machine with the VM to profile, create a policy file which contains the following

// create a policy file called all.policy

grant codebase "file:${java.home}/../lib/tools.jar" {

   permission java.security.AllPermission;

};

Then create a RMI instance on a particular local port (1099 is standard but any port can be used such as 2020)

// Start the rmiregistry

rmiregistry 1099&

And now use jstatd to populate the registry.

// Start jstatd with the policy file above and point it to the rmiregistry

jstatd -J-Djava.security.policy=all.policy -p 1099

Now start jvisualvm on a local machine and connect to port 1099 on the remote machine to view the details.

Within jvisualvm right click on 'Remote' and select 'Add remote host'.  In the hostname add the IP address of the remote machine and in 'Advanced' the port can be change if 1099 isn't used for the rmiregistry or jstatd.

It isn't possible to do everything with a remote connection that you can do with a local connection. For this reason it can be useful to run jvisualvm locally but use X11 forwarding to put the actual application on a remote machine.

Troubleshooting

Sometimes there are connection issues when trying to connect to a remove jstatd.  So check the following

Logs
Look at the jvisualvm logs which can be found in ~/.visualvm/<version>/var/log

Headless
Although jvisualvm can be run with X11 forwarding so it can be a headless environment it has dependencies on some X11 libraries so needs to have a desktop manager installed even if it isn't running. (libXext.so.6)

Firewall
Make sure that any firewall isn't blocking the RMI ports.

RMI address
Sometimes the RMI address needs to be set so try adding

    -J-Djava.rmi.server.hostname=<ip address>

IP version
It is also possible to try to force IPv4 to see if that helps,

    -J-Djava.net.preferIPv4Stack=true 

Other Tools

There are a number of other tools which are useful to ascertaining if there is a memory leak in the heap.  A heap dump can be generated using the jmap command which is part of the jdk

    ./jmap -dump:format=b,file=<filepath> <pid>

This generates a hprof format.  There is a free tool called the Memory Analyser Tool (MAT) which is based on the eclipse platform and can be used to analyse the heap file.

    http://www.eclipse.org/mat/

@Transactional

When connecting to a database a transaction is often required.  There are a number of ways of doing this but the most common is to use Spring transactions.

pom.xml

This dependency is needed to allow transactions

<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-tx</artifactId>
<scope>compile</scope>
</dependency>

Spring Context

In the spring configuration an entity manager is necessary (see Entity Manager blog entry)

<tx:annotation-driven transaction-manager="transactionManager" proxy-target-class="true" />

<!-- Define a transaction manager so that the @TransactionConfiguration and @Transactional can be used -->
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>

Proxy Class

The above combination will allow spring to create proxy classes which create a transaction before passing into the @Transactional method.  This means that because it is a proxy @Transactional doesn't work for any 'internal' call within the your class.  Eg even if a method is public if it is called from within the same class it won't pass through the proxy and therefore won't have a transaction.

If this type of behaviour is required then the transactions can be wired using AOP and the transaction code is created at compile time rather than a runtime proxy.

Exception Handling

Normal exceptions can be caught using the normal try - catch but if the database isn't there at all then there is an exception generated in the proxy before the method code is called.  The exception is a 

    org.springframework.transaction.CannotCreateTransactionException

To catch the exception within the same method as the actual database errors the transaction can be created a different way.

Programmatic Transactions

A programmatic transaction can be created using the TransactionTemplate in which case the TransactionException (CannotCreateTransactionException) can be caught in the same place.

    try

    {

        transactionTemplate.execute(new TransactionCallbackWithoutResult()

        {

            @Override

            protected void doInTransactionWithoutResult(final TransactionStatus status)

            {

                 // Do something to the database here

            }

        });

    }

    catch (PersistenceException | TransactionException e)

    {

        LOG.error("Database Error", e);

    }

There is also a TransactionCallback() which returns the object from the database whereas the one above doesn't do that.

The transactionTemplate can be created in spring configuration,

    <property name="transactionTemplate">

        <bean class="org.springframework.transaction.support.TransactionTemplate">

            <constructor-arg ref="transactionManager" />

</bean>

    </property>

Spring Web App Template

I don't create new web apps that often and therefore can easily forget the necessary bits!  This is my memory jogger for how do to these.

Folders
Notes: In eclipse create a new java project with the correct maven folder structure.  This is the sample folder structure

    project_home/src
    project_home/src/main

    project_home/src/main/java

    project_home/src/main/resources

    project_home/src/main/webapp

    project_home/src/main/webapp/WEB-INF

    project_home/src/main/webapp/WEB-INF/jsp

    project_home/src/test

    project_home/src/test/java

    project_home/src/test/resources

pom.xml

Location: project_home/pom.xml

Notes: The basic maven pom which has the correct values for the spring classes although the spring versions can be updated. 

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>

    <groupId>com.me.app</groupId>

    <artifactId>me-web</artifactId>

    <packaging>war</packaging>

    <version>0.0.1-SNAPSHOT</version>

    <name>Sample Web App</name>

    <description />

    <properties>

        <spring.version>3.1.2.RELEASE</spring.version>

    </properties>

    <dependencies>

        <dependency>

            <groupId>org.springframework</groupId>

            <artifactId>spring-web</artifactId>

            <version>${spring.version}</version>

        </dependency>

        <dependency>

            <groupId>org.springframework</groupId>

            <artifactId>spring-webmvc</artifactId>

            <version>${spring.version}</version>

        </dependency>

        <!-- test -->

        <dependency>

            <groupId>org.springframework</groupId>

            <artifactId>spring-test</artifactId>

            <version>${spring.version}</version>

            <scope>test</scope>

        </dependency>

    </dependencies>

</project>

Eclipse

Notes: After putting this pom in place convert to a maven project.  In eclipse right click on the project and go to Configure - Convert to Maven Project.  This should create a folder called Maven Dependencies which contains all the libraries defined by the pom.

Right click on the project and go to properties.  

 - Select 'Project Facets' and make sure it is a faceted form with Dynamic Web Module, Java and JavaScript are all checked.

 - Click on Deployment Assembly and make sure that Source /src/main/webapp is mapped to Deploy Path / and also that Source: Maven Dependencies are mapped to Deploy Path /WEB-INF/lib.

 - Click on Web Project Settings and check the context-root value

 - Click on Java Build Path and add main/java, main/resources, test/java and test/resources as source folders.

web.xml

Location: project_home/src/main/webapp/WEB-INF/web.xml

Notes: This is the file that is read by the application server to determine details about the servlet it needs to run.  It contains the main servlet class and mappings of which requests get sent to this servlet.  In this case the servlet is the spring MessageDispatchServlet as this is a spring app.

   

 <web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  

    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee   

          http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"  

    version="2.5">

  <display-name>Web App</display-name>

  

  <servlet>  

        <servlet-name>webapp</servlet-name>  

        <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>  

        <load-on-startup>1</load-on-startup>  

    </servlet>  

  

    <servlet-mapping>  

        <servlet-name>webapp</servlet-name>  

        <url-pattern>/app/*</url-pattern>  

    </servlet-mapping>  

  

</web-app>

servlet.xml

Location: project_home/src/mainwebapp/WEB-INF/servlet.xml

Notes: The servlet xml is the file that is used by the spring MessageDispatcherServlet and is the root of the spring configuration for the web application.

<beans xmlns="http://www.springframework.org/schema/beans"  

    xmlns:context="http://www.springframework.org/schema/context"  

    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  

    xsi:schemaLocation="  

        http://www.springframework.org/schema/beans       

        http://www.springframework.org/schema/beans/spring-beans-3.0.xsd  

        http://www.springframework.org/schema/context   

        http://www.springframework.org/schema/context/spring-context-3.0.xsd">  

   

    <context:component-scan base-package="com.me.app.web" />  

   

    <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">  

        <property name="prefix">  

            <value>/WEB-INF/jsp/</value>  

        </property>  

        <property name="suffix">  

            <value>.jsp</value>  

        </property>  

    </bean>  

   

</beans> 

Controller

Location: project_home/src/main/java/com/me/app/web/controller/Greeting.java

Notes: This is a sample controller that maps to a jsp.

package com.me.app.web.controller;

import org.springframework.stereotype.Controller;

import org.springframework.web.bind.annotation.RequestMapping;

import org.springframework.web.servlet.ModelAndView;

/**

 * Sample controller.

 */

@Controller

@RequestMapping("/greeting")

public class Greeting

{

    /**

     * Method to say hello.

     */

    @RequestMapping("/hello")

    public ModelAndView hello()

    {

        final ModelAndView mav = new ModelAndView("hello");

     return mav;

    }

}

jsp

Location: project_home/src/webapp/WEB-INF/jsp/hello.jsp

Notes: This is a very simple jsp file that is used to just smoke test that the application is running.

<html>  

  <head>

    <title>Sample Web App</title>

  </head>  

  <body>

    <center>  

      <h2>Sample Web App</h2>  

      <h4>Hello</h4>  

    </center>  

  </body>  

</html>  

Dynamically Updating JavaFX Table

There are many tricks on the web for updating tables when a value changes and the table doesn't update such as adding and removing columns to force the table to refresh.  I think there may be a refresh table option in Java 8 now.  However, getting the data structure correct makes a big difference.

Using Properties
Use properties in the backing object so that the table can bind to them and get the updates.  In the example below the name is immutable so doesn't need a property but the count is changeable so use a property value and importantly include the countProperty() method.


public class MyTableData
{
    /**
     * A value that doesn't change so just stored as a string.
     */
    private final String name;
        
    /**
     * A changeable value so stored as a property.
     */
    private final SimpleIntegerProperty count;
    
    /**
     * Constructs a new MyTableData with the given parameters.
     *
     * @param name The name
     * @param count The count
     */
    public MyTableData (final String name, final Integer count)
    {
        this.name= name;
        this.count = new SimpleIntegerProperty(count);
    }

    /**
     * Gets the namevalue.
     *
     * @return the name
     */
    public String getName()
    {
        return name;
    }

    /**
     * Gets the count value.
     *
     * @return the count
     */
    public int getCount()
    {
        return count.get();
    }

    /**
     * Set the count value.
     *     * @param count The new value to set
     */
    public void setCount(final int count)
    {
        this.count.set(count);
    }

    /**
     * The count property.
     *
     * @return The count property
     */
    public IntegerProperty countProperty() 
    {
        return count;
    }
}

The Table
In the table create the columns as normal,

        // Count Column
        final TableColumn<MyTableData, Number> countCol = new TableColumn<MyTableData, Number>("Count");
        countCol.setCellValueFactory(new PropertyValueFactory<MyTableData, Number>("count"));

Spring Context Unit Testing

It is always a good idea to unit test the spring contexts that define an application.  This can be as simple as a unit test

    @RunWith(SpringJUnit4ClassRunner.class)
    @ContextConfiguration(locations = { "classpath:com/me/my_context.xml" })
    public class SpringContextTest implements ApplicationContextAware
    {
        /**
         * The spring context.
         */
        private ApplicationContext applicationContext;

        /**
         * Test the spring wiring.
         */
        @Test
        public void testSpringWiring()
        {
            final Object obj = applicationContext.getBean("myBean");
            assertTrue(obj instanceof MyBean);
        }

        /**
         * {@inheritDoc}
         */
        @Override
        public void setApplicationContext(final ApplicationContext applicationContext)
        {
            this.applicationContext = applicationContext;
        }
    }


However, where there is interaction with a JNDI resource as part of the spring start up this needs to be modified to make the JNDI resource available.  Adding this @Before method will set up a JNDI resource for a MySql database.

    /**
     * Set up the jndi data source so that the wiring tests still work.
     */
    @BeforeClass
    public static void setUpClass()
    {
        // Setup the jndi context and the datasource
        try
        {
            // Create a database connection to satisfy the data loading requirements.
            final DriverManagerDataSource dataSource = new DriverManagerDataSource();
            dataSource.setDriverClassName("com.mysql.jdbc.Driver");
            dataSource.setUrl("jdbc:mysql://server:port");
            dataSource.setUsername("username");
            dataSource.setPassword("password");

            final SimpleNamingContextBuilder builder = SimpleNamingContextBuilder.emptyActivatedContextBuilder();
            builder.bind("jdbc/MY_JNDI_NAME", dataSource);
            builder.activate();
        }
        catch (final NamingException ex)
        {
            LOG.error("Error creating JNDI resource", ex);
        }
    }

Command Line Application Start

Java has the 'main' method to start its apps and this is very easy to invoke.  However, there are times when optional arguments need to be passed into the app and for this the apache commons cli (Command Line Interface) is really useful.

Creating a Parser

Creating a parser is really simple and involves just one line

    CommandLineParser parser = new BasicParser();

Creating Options

The options for the command line are built using a builder pattern.  Here is an example

    OptionBuilder
        // the name of the argument
        .withArgName("timeout") 
        // this has two arguments -timeout <value>
        .hasArgs(2)
        // the separator '-' is used
        .withValueSeparator()
        // the description used for the help text
        .withDescription("Millisecond timeout for this test (default: 20000)")
        // create this option refered to as 'timeout'      
        .create("timeout")

This option is then added to the list of options

    final Options options = new Options();
    options.addOption(OptionBuilder.withArgName(.......));

which are added to the parser

    CommandLine line = parser.parse(options, args);

Getting the values

This CommandLine object can then be used to get the values that have been provided.

    final String timeoutStr = line.getOptionValue("timeout", "20000");

or query that an option has been provided,

    if (line.hasOption("timeout")) {
    
    }

Help Text
The descriptions created in the Options can be used to generate help text

    final HelpFormatter formatter = new HelpFormatter();
    formatter.printHelp("sample", options);

which is useful if a value fails validation or particular options are not provided.

Example

Here is a full example

import org.apache.commons.cli.BasicParser;
import org.apache.commons.cli.CommandLine;
import org.apache.commons.cli.CommandLineParser;
import org.apache.commons.cli.HelpFormatter;
import org.apache.commons.cli.OptionBuilder;
import org.apache.commons.cli.Options;
import org.apache.commons.cli.ParseException;

public class SampleMain
{

    /**
     * Main.
     * 
     * @param args The command line args
     */
    public static void main(final String[] args)
    {

        final CommandLineParser parser = new BasicParser();
        final Options options = new Options();
        options.addOption(OptionBuilder.withArgName("timeout").hasArgs(2).withValueSeparator()
                .withDescription("Millisecond timeout for this app (default: 20000)").create("timeout"));
        options.addOption(OptionBuilder.withArgName("filepath").hasArgs(2).withValueSeparator()
                .withDescription("Configuration file").create("filepath"));
        options.addOption(OptionBuilder.withArgName("logdirectory").hasArgs(2).withValueSeparator()
                .withDescription("log directory (default: ./logs)").create("logdirectory"));
        
        CommandLine line = parser.parse(options, args);
        
        // Get the timeout
        final String timeoutStr = line.getOptionValue("timeout", "20000");
        final long timeout = Long.valueOf(timeoutStr);

        // Log directory for the app
        final String logDirectory = line.getOptionValue("logdirectory", "./logs");
        
        if (line.hasOption("filepath"))
        {
            // Do something with this file
        }
        else
        {
            final HelpFormatter formatter = new HelpFormatter();
            formatter.printHelp("sample", options);
        }
    }
}

Use

This code would then be used,

    java -jar sample.jar -logdirectory ./mylog -timeout 5000 -filepath mydir/myfile

or to use the defaults

    java -jar sample.jar -filepath mydir/myfile

Dynamic Spring Message Driven Beans

Creating MDBs from spring is really easy particularly if all the information is known up front.  As usual it is just some configuration in the spring context and is shown here.

For dynamic beans things can still be put into the spring configuration but need the destination setting at runtime.  This can be done using some spring context as prototype beans so that a new instance of them is created each time and a small bit of wiring in the code.

Spring Context

<!-- MQ / JMS Connection factory -->

<bean id="jmsConnectionFactory" class="com.ibm.mq.jms.MQTopicConnectionFactory">

<property name="hostName" value="${jms.mq.ip}" />

<property name="port" value="${jms.mq.port}" />

<property name="queueManager" value="${jms.mq.queueManager}" />

<property name="transportType" value="1" />

</bean>

<!-- Spring Connection Factory -->

<bean id="springConnectionFactory" class="org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter">

<property name="targetConnectionFactory" ref="jmsConnectionFactory" />

<property name="username" value="${jms.mq.username}" />

<property name="password" value="${jms.mq.password}" />

</bean>

<!-- This bean is a prototype bean which is loaded dynamically-->
<!-- it is loaded using a getBean("jmsDestination", requiredTopic") and as such the HOLDING_VALUE is -->
<!-- replaced with whatever is required by the class in question                                    -->
<bean id="jmsDestination" class="com.ibm.mq.jms.MQTopic" scope="prototype">
<constructor-arg value="HOLDING_VALUE" />
</bean>

<!-- Generic MDB Container -->
<bean id="springMdbContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer" scope="prototype">
<property name="connectionFactory" ref="springConnectionFactory" />
<!-- The destination is dynamically set in the factory method but a default is provided here-->
<property name="destination" ref="jmsDestination" />
<!-- The normal MessageListener which is put here is added dynamically in the factory -->
</bean>

Java

In the java code a new destination can be obtained using the bean above and the code,

    /**
     * Create a Destination object.
     *
     * @param topic The topic to construct the destination with
     * @return the Destination created.
     */
    public Destination createDestination(final String topic)
    {
        // Create the destination with the topic string.
        return (Destination) applicationContext.getBean("jmsDestination", topic);
    }

This creates a new Destination object based on the spring bean.  This allows the MQ provided to be changed in the spring configuration without having to change code.

To create a new MDB create a destination as above and then use the code,
   
    /**
     * Create a MDB wired by spring.
     *
     * @param destination The destination topic to listen to.
     * @param messageListener The message listener (MDB) to wire
     */
    private void wireMdb(final Destination destination, final MessageListener messageListener)
    {
        // Use spring to wire up the MDB with this destination and a ConnectionFactory 
        // which is already provided in the spring context. 
        // The MDB and Destination are set here and then the container is manually started.
        final DefaultMessageListenerContainer container = 
                (DefaultMessageListenerContainer) applicationContext.getBean("springMdbContainer");
        container.setDestination(destination);
        container.setMessageListener(messageListener);
        container.start();
    }

It is also possible to change the destination that a container is listening to once it has started.  Using the setDestination(...) method does this according to the spring documentation.