| PolarSPARC |
Dynamic Code Generation using Java Compiler API
| Bhaskar S | *UPDATED*01/24/2025 |
Overview
In one of the previous articles, we were introduced to Scripting in Java. It was one way of dynamically extending the capabilities of an application. Also, there is another efficient way of extending an application dynamically - by compiling and loading Java classes at runtime.
In Java, the javax.tools package exposes the Java compiler as an API. By default, the Java compiler works with source from input file(s) and generates the corresponding class output file(s). By implementing interfaces from the javax.tools package, we will be able to work with source from strings in memory and generate class to byte array in memory.
But, why would we need to do this ? Imagine we have an asynchronous channel from where clients can consume messages. Each client may have a different need and process only a subset of the messages. The clients effectively need to filter on the content of the messages before processing. If the filter criteria for all the clients is known and is a small predefined set, then we may be able to write the filter classes for the predefined set. But, if the filter criteria various for each client and is subject to change, it is better to implement the filter criteria as a dynamic extension. If the message rate is small and the filter criteria is not that complex, then we may be able to leverage the scripting support in Java. But, if the message rates are high and the filter criteria complex, then scripting may not be a viable option for efficiency and performance reasons. It would be more efficient to dynamically compile and execute the filter criteria.
Setup
The setup will be on a Ubuntu 24.04 LTS based Linux desktop. Ensure at least Java 17 or above is installed and setup. Also, ensure Apache Maven is installed and setup.
To setup the Java directory structure for the demonstrations in this article, execute the following commands:
$ cd $HOME
$ mkdir -p $HOME/java/JavaCompiler
$ cd $HOME/java/JavaCompiler
$ mkdir -p src/main/java src/main/resources target
$ mkdir -p src/main/java/com/polarsparc/jdk/compiler
$ mkdir -p src/main/resources/scripts
The following is the listing for the Maven project file pom.xml that will be used:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.polarsparc.javacompiler</groupId>
<artifactId>JavaCompiler</artifactId>
<version>1.0</version>
<properties>
<java.version>23</java.version>
<slf4j.version>2.0.16</slf4j.version>
<nashorn.version>15.6</nashorn.version>
<maven.compiler.version>3.13.0</maven.compiler.version>
</properties>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven.compiler.version}</version>
<configuration>
<fork>true</fork>
<meminitial>128m</meminitial>
<maxmem>512m</maxmem>
<source>${java.version}</source>
<target>${java.version}</target>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
<dependencies>
<dependency>
<groupId>org.openjdk.nashorn</groupId>
<artifactId>nashorn-core</artifactId>
<version>${nashorn.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>${slf4j.version}</version>
</dependency>
</dependencies>
</project>
The following is the listing for the slf4j-simple logger properties file simplelogger.properties located in the directory src/main/resources:
# ### SLF4J Simple Logger properties # org.slf4j.simpleLogger.defaultLogLevel=info org.slf4j.simpleLogger.showDateTime=true org.slf4j.simpleLogger.dateTimeFormat=yyyy-MM-dd HH:mm:ss:SSS org.slf4j.simpleLogger.showThreadName=true
Without much further ado, let us jump right into hands-on coding and demonstrations.
Before we proceed further, we want to state that in our examples we will be using an array of integers to represent messages and define an interface for the filter criteria.
The following code listing defines the interface for the filter criteria:
/*
* Name: Filter
* Author: Bhaskar S
* Date: 01/18/2025
* Blog: https://www.polarsparc.com
*/
package com.polarsparc.jdk.compiler;
public interface Filter {
public boolean filter(int num);
}
The following code listing illustrates an implementation of the filter criteria where the input integer is an even integer and is greater than 500:
/*
* Name: Filter
* Author: Bhaskar S
* Date: 01/18/2025
* Blog: https://www.polarsparc.com
*/
package com.polarsparc.jdk.compiler;
public class FilterEvenGt500 implements Filter {
@Override
public boolean filter(int num) {
return num > 500 && (num % 2) == 0;
}
}
The following code listing for ScriptVsCompiled illustrates an example that compares the efficiency of using scripting versus compiled code:
/*
* Name: ScriptVsCompiled
* Author: Bhaskar S
* Date: 01/18/2025
* Blog: https://www.polarsparc.com
*/
package com.polarsparc.jdk.compiler;
import java.util.*;
import javax.script.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class ScriptVsCompiled {
private static final Logger LOGGER = LoggerFactory.getLogger(ScriptVsCompiled.class);
private static int[] nums;
public static void main(String[] args) {
init();
ScriptClient sc = new ScriptClient();
sc.init();
sc.execute();
CompiledClient cc = new CompiledClient();
cc.init();
cc.execute();
}
public static void init() {
try {
nums = new int[100];
Random rand = new Random();
for (int i = 0; i < 100; ++i) {
nums[i] = rand.nextInt(1000);
}
}
catch (Throwable ex) {
ex.printStackTrace(System.err);
System.exit(1);
}
}
private static class ScriptClient {
private ScriptEngine seng;
void init() {
ScriptEngineManager smgr = new ScriptEngineManager();
seng = smgr.getEngineByExtension("js");
if (seng == null) {
LOGGER.error("*** Could not find engine - Rhino !!!");
System.exit(1);
}
}
void execute() {
try {
Bindings bnds = seng.getBindings(ScriptContext.ENGINE_SCOPE);
long stm = System.currentTimeMillis();
for (int n : nums) {
bnds.put("num", n);
if ((Boolean)seng.eval("if (num > 500 && (num % 2) == 0) { true; } else { false; }")) {
LOGGER.info("ScriptClient: {}", n);
}
}
long etm = System.currentTimeMillis();
LOGGER.info("ScriptClient execute time: {} ms", (etm - stm));
}
catch (Throwable ex) {
LOGGER.error(ex.getMessage(), ex);
System.exit(1);
}
}
}
private static class CompiledClient {
private Filter func;
void init() {
try {
Class<?> clazz = Class.forName("com.polarsparc.jdk.compiler.FilterEvenGt500");
func = (Filter) clazz.getDeclaredConstructor().newInstance();
}
catch (Throwable ex) {
LOGGER.error(ex.getMessage(), ex);
System.exit(1);
}
}
void execute() {
try {
long stm = System.currentTimeMillis();
for (int n : nums) {
if (func.filter(n)) {
LOGGER.info("CompiledClient: {}", n);
}
}
long etm = System.currentTimeMillis();
LOGGER.info("CompiledClient execute time: {} ms", (etm-stm));
}
catch (Throwable ex) {
LOGGER.error(ex.getMessage(), ex);
System.exit(1);
}
}
}
}
To execute the code from Listing.3, open a terminal window and run the following commands:
$ cd $HOME/java/JavaCompiler
$ mvn exec:java -Dexec.mainClass="com.polarsparc.jdk.compiler.ScriptVsCompiled"
The following would be the typical output:
[INFO] Scanning for projects... [INFO] [INFO] --------------< com.polarsparc.javacompiler:JavaCompiler >-------------- [INFO] Building JavaCompiler 1.0 [INFO] from pom.xml [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- exec:3.5.0:java (default-cli) @ JavaCompiler --- 2025-01-18 21:06:09:089 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 642 2025-01-18 21:06:09:090 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 736 2025-01-18 21:06:09:091 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 746 2025-01-18 21:06:09:091 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 834 2025-01-18 21:06:09:092 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 760 2025-01-18 21:06:09:092 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 564 2025-01-18 21:06:09:092 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 956 2025-01-18 21:06:09:093 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 594 2025-01-18 21:06:09:093 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 968 2025-01-18 21:06:09:093 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 692 2025-01-18 21:06:09:094 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 934 2025-01-18 21:06:09:094 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 594 2025-01-18 21:06:09:094 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 532 2025-01-18 21:06:09:094 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 630 2025-01-18 21:06:09:094 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 828 2025-01-18 21:06:09:095 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 628 2025-01-18 21:06:09:096 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 552 2025-01-18 21:06:09:096 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 692 2025-01-18 21:06:09:097 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 956 2025-01-18 21:06:09:097 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 728 2025-01-18 21:06:09:097 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 950 2025-01-18 21:06:09:097 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 714 2025-01-18 21:06:09:097 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 878 2025-01-18 21:06:09:098 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 950 2025-01-18 21:06:09:098 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient: 578 2025-01-18 21:06:09:099 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - ScriptClient execute time: 61 ms 2025-01-18 21:06:09:099 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 642 2025-01-18 21:06:09:099 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 736 2025-01-18 21:06:09:099 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 746 2025-01-18 21:06:09:099 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 834 2025-01-18 21:06:09:099 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 760 2025-01-18 21:06:09:099 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 564 2025-01-18 21:06:09:099 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 956 2025-01-18 21:06:09:099 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 594 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 968 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 692 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 934 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 594 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 532 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 630 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 828 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 628 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 552 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 692 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 956 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 728 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 950 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 714 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 878 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 950 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient: 578 2025-01-18 21:06:09:100 [com.polarsparc.jdk.compiler.ScriptVsCompiled.main()] INFO com.polarsparc.jdk.compiler.ScriptVsCompiled - CompiledClient execute time: 1 ms [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 0.472 s [INFO] Finished at: 2025-01-18T21:06:09-05:00 [INFO] ------------------------------------------------------------------------
As is evident from Output.1 above, using scripting capability of Java for filtering is orders of magnitude slower than using compiled code for filtering !!!
The client using the scripting capability (named ScriptClient) took about 61 ms to complete, while the client (name CompiledClient) using the compiled filter class FilterEvenGt500 took 1 ms.
It is clear from the above execution that it will be more efficient to dynamically compile and execute the filter criteria.
Ready for some exciting bytecode stuff !!! In the following sections we will explore the Java compiler API exposed through the javax.tools package in Java.
The following code listing is a simple Java class called CompileMe that we will compile using the Java compiler API:
/*
* Name: CompileMe
* Author: Bhaskar S
* Date: 01/18/2025
* Blog: https://www.polarsparc.com
*/
package com.polarsparc.jdk.compiler;
public class CompileMe {
public static void main(String[] args) {
System.out.println("Compiled with JavaCompiler API");
}
}
The following code listing for BasicCompile illustrates the use of the Java compiler API to compile the simple Java class called CompileMe:
/*
* Name: BasicCompile
* Author: Bhaskar S
* Date: 01/18/2025
* Blog: https://www.polarsparc.com
*/
package com.polarsparc.jdk.compiler;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.*;
import javax.tools.*;
import javax.tools.JavaCompiler.CompilationTask;
public class BasicCompile {
private static final Logger LOGGER = LoggerFactory.getLogger(BasicCompile.class);
public static void main(String[] args) {
try {
/* 1 */
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
/* 2 */
StandardJavaFileManager manager = compiler.getStandardFileManager(null, null, null);
/* 3 */
Iterable<? extends JavaFileObject> units = manager.getJavaFileObjects("src/main/java/com/polarsparc/jdk/compiler/CompileMe.java");
/* 4 */
String[] opts = new String[] { "-d", "target/classes" };
/* 5 */
CompilationTask task = compiler.getTask(null, manager, null, Arrays.asList(opts), null, units);
/* 6 */
boolean status = task.call();
if (status) {
LOGGER.info("Compilation successful!!!");
}
}
catch (Exception ex) {
LOGGER.error(ex.getMessage(), ex);
System.exit(1);
}
}
}
In line /* 1 */, we get the reference to the implementation of the Java platform compiler called JavaCompiler.
In line /* 2 */, we get the reference to StandardJavaFileManager, which provides an abstraction layer for performing file operations such as reading an input source and writing compiled class output. Just as with the javac command, the JavaCompiler also operates on file(s) using StandardJavaFileManager.
In line /* 3 */, we wrap the simple Java source file called CompileMe.java in a JavaFileObject. The StandardJavaFileManager works on the input and the output files and provides them as objects of type JavaFileObject.
In line /* 4 */, we are specifying the compiler options to use. In this case, we are specifying the target directory of compile to be the dircetory "target/classes". By default, the target directory is the current directory.
In line /* 5 */, we get the reference to CompilationTask, which allows us to invoke the process of Java source compilation. We get an handle to CompilationTask by invoking the getTask() method on the reference to JavaCompiler from line /* 1 */. In this step, we associate the StandardJavaFileManager, the compiler options, and the input source file wrapped in JavaFileObject.
In line /* 6 */, we finally invoke the compilation task. If the compilation succeeds, the compiled class file will be under the directory named "target/classes".
To execute the code from Listing.5, open a terminal window and run the following commands:
$ cd $HOME/java/JavaCompiler
$ mvn exec:java -Dexec.mainClass="com.polarsparc.jdk.compiler.BasicCompile"
The following would be the typical output:
[INFO] Scanning for projects... [INFO] [INFO] --------------< com.polarsparc.javacompiler:JavaCompiler >-------------- [INFO] Building JavaCompiler 1.0 [INFO] from pom.xml [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- exec:3.5.0:java (default-cli) @ JavaCompiler --- 2025-01-18 21:30:51:996 [com.polarsparc.jdk.compiler.BasicCompile.main()] INFO com.polarsparc.jdk.compiler.BasicCompile - Compilation successful!!! [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 0.501 s [INFO] Finished at: 2025-01-18T21:30:51-05:00 [INFO] ------------------------------------------------------------------------
With this, we have successfully demonstrated the use of the Java compiler API to generate java class file from a java source file.
In the above example, the java code was sourced from a file. How do we handle the case where the java code is generated dynamically at runtime as a String. One way would be to save the generated code to a file and then use the above example to compile. The more interesting case would be to compile the code directly from the String.
In the above example, we used StandardJavaFileManager to wrap the java source file as an object of type JavaFileObject. Similarly, we need a class to wrap the generated code from a String into an object of type JavaFileObject. We can achieve that by extending the concrete class SimpleJavaFileObject from the javax.tools package.
The following code listing for StringJavaFileObject illustrates our custom JavaFileObject that allows us to present java code from a String to the JavaCompiler for compilation:
/*
* Name: StringJavaFileObject
* Author: Bhaskar S
* Date: 01/18/2025
* Blog: https://www.polarsparc.com
*/
package com.polarsparc.jdk.compiler;
import java.net.*;
import javax.tools.*;
/* 1 */
public class StringJavaFileObject extends SimpleJavaFileObject {
private final String source;
/* 2 */
public StringJavaFileObject(String name, String source) {
/* 3 */
super(URI.create("string:///" + name.replace('.','/') + Kind.SOURCE.extension), Kind.SOURCE);
this.source = source;
}
/* 4 */
@Override
public CharSequence getCharContent(boolean ignoreEncodingErrors) {
return this.source;
}
}
In line /* 1 */, we extend from the class SimpleJavaFileObject, which is provided by the Java Compiler API as a simple implementation of JavaFileObject. The constructor for SimpleJavaFileObject is defined as protected and takes two arguments: an URI to the file it represents and the type of the file (java source or compiled class) specified as a constant of type Kind.
In line /* 2 */, we define the constructor for our custom JavaFileObject implementation called StringJavaFileObject. It takes two arguments: the full class name (including package name) for our generated java source and the java code as a String.
In line /* 3 */, we invoke the constructor for the super class which is SimpleJavaFileObject in this case. The line URI.create("string:///" + name.replace('.','/') + Kind.SOURCE.extension) creates a standard URL path. For example, given the full class name "com.abc.Foo", this line will create a standard URL path "com/abc/Foo.java". The enum Kind (defined in JavaFileObject) identifies the type of the URI. In our case, the type is java source and hence we use Kind.SOURCE.
In line /* 4 */, we override the method getCharContent to return the java source from the String.
The following code listing for StringCompile illustrates the use of our custom StringJavaFileObject to compile and execute a Filter class implementation that is generated as a String:
/*
* Name: StringCompile
* Author: Bhaskar S
* Date: 01/18/2025
* Blog: https://www.polarsparc.com
*/
package com.polarsparc.jdk.compiler;
import java.util.*;
import java.io.*;
import java.net.*;
import javax.tools.*;
import javax.tools.JavaCompiler.CompilationTask;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class StringCompile {
private static final Logger LOGGER = LoggerFactory.getLogger(StringCompile.class);
public static void main(String[] args) {
try {
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
StandardJavaFileManager manager = compiler.getStandardFileManager(null, null, null);
/* 1 */
JavaFileObject strFile = new StringJavaFileObject("com.polarsparc.jdk.compiler.FilterEven", generateJavaCode());
Iterable<? extends JavaFileObject> units = List.of(strFile);
String[] opts = new String[] { "-d", "target/classes", "-classpath", "target/classes" };
CompilationTask task = compiler.getTask(null, manager, null, Arrays.asList(opts), null, units);
boolean status = task.call();
if (status) {
LOGGER.info("Compilation successful!!!");
File classesDir = new File("classes");
URL[] classpath = new URL[] { classesDir.toURI().toURL() };
URLClassLoader urlClassloader = null;
try {
/* 2 */
urlClassloader = new URLClassLoader(classpath, StringCompile.class.getClassLoader());
/* 3 */
final Class<?> clazz = urlClassloader.loadClass("com.polarsparc.jdk.compiler.FilterEven");
/* 4 */
final Filter filter = (Filter) clazz.getDeclaredConstructor().newInstance();
/* 5 */
if (filter.filter(10)) {
LOGGER.info("10 is an even number");
}
/* 6 */
if (!filter.filter(15)) {
LOGGER.info("15 is an odd number");
}
}
finally {
if (urlClassloader != null) {
urlClassloader.close();
}
}
}
else {
LOGGER.error("***** Compilation failed!!!");
}
}
catch (Exception ex) {
LOGGER.error(ex.getMessage(), ex);
System.exit(1);
}
}
private static String generateJavaCode() {
return
"""
package com.polarsparc.jdk.compiler;
import com.polarsparc.jdk.compiler.Filter;
public class FilterEven implements Filter {
@Override
public boolean filter(int num) {
if (num % 2 == 0) {
return true;
}
return false;
}
}
""";
}
}
In line /* 1 */, we create an instance of StringJavaFileObject by specifying the full class name as com.polarsparc.jdk.compiler.FilterEven and generating the java source for FilterEven by calling the method generateJavaCode.
The steps to the compile the generated code is the same as was illustrated in BasicCompile. Once the compilation proceeds successfully, a class file for FilterEven is generated under the target/classes directory.
In line /* 2 */, we create an instance of URLClassLoader for the target/classes directory so that we can load the dynamically generated compiled class for FilterEven.
In line /* 3 */, we load the class for FilterEven using the URLClassLoader created in /* 2 */.
In line /* 4 */, we create an instance of FilterEven, which implements the interface for Filter.
In line /* 5 */ and /* 6 */, we invoke the method filter on the FilterEven instance created in /* 4 */.
To execute the code from Listing.7, open a terminal window and run the following commands:
$ cd $HOME/java/JavaCompiler
$ mvn exec:java -Dexec.mainClass="com.polarsparc.jdk.compiler.StringCompile"
The following would be the typical output:
[INFO] Scanning for projects... [INFO] [INFO] --------------< com.polarsparc.javacompiler:JavaCompiler >-------------- [INFO] Building JavaCompiler 1.0 [INFO] from pom.xml [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- exec:3.5.0:java (default-cli) @ JavaCompiler --- 2025-01-19 14:07:33:914 [com.polarsparc.jdk.compiler.StringCompile.main()] INFO com.polarsparc.jdk.compiler.StringCompile - Compilation successful!!! 2025-01-19 14:07:33:915 [com.polarsparc.jdk.compiler.StringCompile.main()] INFO com.polarsparc.jdk.compiler.StringCompile - 10 is an even number 2025-01-19 14:07:33:915 [com.polarsparc.jdk.compiler.StringCompile.main()] INFO com.polarsparc.jdk.compiler.StringCompile - 15 is an odd number [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 0.466 s [INFO] Finished at: 2025-01-19T14:07:33-05:00 [INFO] ------------------------------------------------------------------------
With this, we have successfully demonstrated the use of the Java compiler API to generate java class file from dynamically generated java source from a String.
The next natural question to ask is: what if we make a mistake in the java source generation. Let us change the method generateJavaCode in StringCompile as follows:
private static String generateJavaCode() {
return
"""
package com.polarsparc.jdk.compiler;
import com.polarsparc.jdk.compiler.Filter;
public class FilterEven implements Filter {
@Override
public boolean filter(int num) {
if (num % 2 == 0) {
return 2;
}
return false;
}
}
""";
}
Executing the code from Listing.7 with the change from Listing.8 will result in the following typical output:
[INFO] Scanning for projects...
[INFO]
[INFO] --------------< com.polarsparc.javacompiler:JavaCompiler >--------------
[INFO] Building JavaCompiler 1.0
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- exec:3.5.0:java (default-cli) @ JavaCompiler ---
/com/polarsparc/jdk/compiler/FilterEven.java:7: error: incompatible types: int cannot be converted to boolean
return 2;
^
1 error
2025-01-19 14:12:41:236 [com.polarsparc.jdk.compiler.StringCompile.main()] ERROR com.polarsparc.jdk.compiler.StringCompile - ***** Compilation failed!!!
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.474 s
[INFO] Finished at: 2025-01-19T14:12:41-05:00
[INFO] ------------------------------------------------------------------------
The JavaCompiler API fails to compile the generated java source and provides enough information on the failure.
In the StringCompile example, the compiled class file for the dynamically generated java source is saved under the target/classes directory. It would be most interesting if the compiled class is also generated and loaded from memory via a byte array.
The JavaCompiler uses StandardJavaFileManager to perform both read and write on files using objects of type JavaFileObject. The JavaCompiler reads the input source file and on successful compile writes the output compiled class file. Just as we used the class StringJavaFileObject to wrap the generated java source in a String, we will use a custom class that will extend SimpleJavaFileObject to wrap the compiled class bytes.
The following code listing for ByteArrayJavaFileObject illustrates our custom JavaFileObject that allows the JavaCompiler to write compiled class bytes to a byte array:
/*
* Name: ByteArrayJavaFileObject
* Author: Bhaskar S
* Date: 01/18/2025
* Blog: https://www.polarsparc.com
*/
package com.polarsparc.jdk.compiler;
import java.io.*;
import java.net.*;
import javax.tools.*;
public class ByteArrayJavaFileObject extends SimpleJavaFileObject {
private final ByteArrayOutputStream bos = new ByteArrayOutputStream();
public ByteArrayJavaFileObject(String name, Kind kind) {
super(URI.create("string:///" + name.replace('.', '/') + kind.extension), kind);
}
public byte[] getClassBytes() {
return bos.toByteArray();
}
@Override
public OutputStream openOutputStream()
throws IOException {
return bos;
}
}
The above code listing is similar to that of StringJavaFileObject, except that we are overriding the method openOutputStream so that the JavaCompiler can use it to output compiled class bytes.
As indicated earlier, the StandardJavaFileManager is a default file manager that creates JavaFileObject instances representing regular files from the file system and used by the JavaCompiler. In order to use our StringJavaFileObject for input and ByteArrayJavaFileObject for output, we need to have a custom JavaFileManager. We cannot extend the StandardJavaFileManager as it does not expose any public constructor and is created internally by the JavaCompiler. Instead, we extend the delegating file manager from the javax.tools package called ForwardingJavaFileManager that allows for customization while delegating to the underlying default file manager the StandardJavaFileManager.
Once the JavaCompiler write the compiled class to a byte array, we will need a way to load the bytes into the class loader. In order to do this we will need a custom class loader to load a class from class bytes.
The following code listing for ByteArrayClassLoader illustrates our custom class loader which extends the default java class loader to load the class bytes from the ByteArrayJavaFileObject:
/*
* Name: ByteArrayClassLoader
* Author: Bhaskar S
* Date: 01/18/2025
* Blog: https://www.polarsparc.com
*/
package com.polarsparc.jdk.compiler;
import java.util.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class ByteArrayClassLoader extends ClassLoader {
private static final Logger LOGGER = LoggerFactory.getLogger(ByteArrayClassLoader.class);
private final Map<String, ByteArrayJavaFileObject> cache = new HashMap<String, ByteArrayJavaFileObject>();
public ByteArrayClassLoader()
throws Exception {
super(ByteArrayClassLoader.class.getClassLoader());
}
public void put(String name, ByteArrayJavaFileObject obj) {
cache.putIfAbsent(name, obj);
}
@Override
protected Class<?> findClass(String name)
throws ClassNotFoundException {
Class<?> cls = null;
try {
ByteArrayJavaFileObject co = cache.get(name);
if (co != null) {
byte[] ba = co.getClassBytes();
cls = defineClass(name, ba, 0, ba.length);
}
}
catch (Exception ex) {
throw new ClassNotFoundException("Class name: " + name, ex);
}
LOGGER.info("Method findClass() called for class {}", name);
return cls;
}
}
In the above custom class loader, we maintain an internal cache of ByteArrayJavaFileObject for each of the dynamically generated java source. The cache is update by our custom JavaFileManager for each generated java source.
The following code listing for DynamicClassFileManager illustrates our custom JavaFileManager that will be used by the JavaCompiler to read java source from a String and to write the compiled class to a byte array:
/*
* Name: DynamicClassFileManager
* Author: Bhaskar S
* Date: 01/18/2025
* Blog: https://www.polarsparc.com
*/
package com.polarsparc.jdk.compiler;
import java.io.*;
import javax.tools.*;
import javax.tools.JavaFileObject.Kind;
/* 1 */
public class DynamicClassFileManager <FileManager> extends ForwardingJavaFileManager<JavaFileManager> {
private ByteArrayClassLoader loader = null;
DynamicClassFileManager(StandardJavaFileManager mgr) {
super(mgr);
try {
/* 2 */
loader = new ByteArrayClassLoader();
}
catch (Exception ex) {
ex.printStackTrace(System.out);
}
}
/* 3 */
@Override
public JavaFileObject getJavaFileForOutput(Location location, String name, Kind kind, FileObject sibling)
throws IOException {
ByteArrayJavaFileObject co = new ByteArrayJavaFileObject(name, kind);
loader.put(name, co);
return co;
}
/* 4 */
@Override
public ClassLoader getClassLoader(Location location) {
return loader;
}
}
In line /* 1 */, we extend from the class ForwardingJavaFileManager which is provided by the Java compiler API as a simple implementation for delegating JavaFileManager. As indicated earlier, we cannot extend SimpleJavaFileManager as it does not expose any public constructor. Instead, we use the delegator ForwardingJavaFileManager, which delegates to the underlying SimpleJavaFileManager.
In line /* 2 */, we create an instance of our custom class loader ByteArrayClassLoader.
In line /* 3 */, we override the method getJavaFileForOutput. This method is invoked by the JavaCompiler when it is ready to write the compiled class bytes for the given java source input. In this method, we create any instance of the custom ByteArrayJavaFileObject and save it in our custom class loader ByteArrayClassLoader for the given class name indicated by the name argument.
In line /* 4 */, we override the method getClassLoader to return our custom class loader ByteArrayClassLoader.
We now have all the necessary ingredients to create any java source and its corresponding class on the fly at runtime. The following code listing for DynamicCompiler puts all these ingredients together:
/*
* Name: DynamicCompiler
* Author: Bhaskar S
* Date: 01/18/2025
* Blog: https://www.polarsparc.com
*/
package com.polarsparc.jdk.compiler;
import java.util.*;
import javax.tools.*;
import javax.tools.JavaCompiler.CompilationTask;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class DynamicCompiler {
private static final Logger LOGGER = LoggerFactory.getLogger(DynamicCompiler.class);
private JavaCompiler compiler;
private DiagnosticCollector<JavaFileObject> collector;
private JavaFileManager manager;
public void init()
throws Exception {
compiler = ToolProvider.getSystemJavaCompiler();
collector = new DiagnosticCollector<JavaFileObject>();
manager = new DynamicClassFileManager<JavaFileManager>(compiler.getStandardFileManager(null, null, null));
}
public Class<?> compileToClass(String fullName, String javaCode)
throws Exception {
Class<?> clazz = null;
StringJavaFileObject strFile = new StringJavaFileObject(fullName, javaCode);
Iterable<? extends JavaFileObject> units = List.of(strFile);
String[] opts = new String[] { "-classpath", "target/classes" };
CompilationTask task = compiler.getTask(null, manager, collector, Arrays.asList(opts), null, units);
boolean status = task.call();
if (status) {
LOGGER.info("Compilation successful!!!");
clazz = manager.getClassLoader(null).loadClass(fullName);
}
else {
LOGGER.info("Message:");
for (Diagnostic<?> d : collector.getDiagnostics()) {
LOGGER.info("{}", d.getMessage(null));
}
LOGGER.info("***** Compilation failed!!!");
}
return clazz;
}
}
The above code provides a convenience method compileToClass that takes a full class name and its corresponding java code and returns the corresponding Class object for that class name. Pay close attention to the code and you will see the DiagnosticCollector. Earlier, we saw the result of wrong code generation. The error was reported by the JavaCompiler in a standard format. If we want customize error reporting, then we can collect the error information by using the DiagnosticCollector.
Finally, we present a demonstration that shows how to use the DynamicCompiler for dynamic code generation. The following code listing for CompiledFilter generates two types of Filter implementation based on the user preference:
/*
* Name: CompiledFilter
* Author: Bhaskar S
* Date: 01/18/2025
* Blog: https://www.polarsparc.com
*/
package com.polarsparc.jdk.compiler;
import java.util.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class CompiledFilter {
private static final Logger LOGGER = LoggerFactory.getLogger(CompiledFilter.class);
private static int[] nums;
private static Filter func;
public static void main(String[] args) {
if (args.length != 1) {
LOGGER.warn("Usage: java {} <DynFilterEvenGt500 | DynFilterOddLt500>", CompiledFilter.class.getName());
System.exit(1);
}
init(args[0]);
execute();
}
public static void init(String name) {
try {
nums = new int[100];
Random rand = new Random();
for (int i = 0; i < 100; ++i) {
nums[i] = rand.nextInt(1000);
}
DynamicCompiler compiler = new DynamicCompiler();
compiler.init();
Class<?> clazz = null;
if (name.equals("DynFilterEvenGt500")) {
clazz = compiler.compileToClass("com.polarsparc.jdk.compiler." + name, DynFilterEvenGt500());
}
else {
clazz = compiler.compileToClass("com.polarsparc.jdk.compiler." + name, DynFilterOddLt500());
}
func = (Filter) clazz.getDeclaredConstructor().newInstance();
}
catch (Throwable ex) {
LOGGER.error(ex.getMessage(), ex);
System.exit(1);
}
}
public static void execute() {
try {
long stm = System.currentTimeMillis();
for (int n : nums) {
if (func.filter(n)) {
LOGGER.info("CompiledFilter: {}", n);
}
}
long etm = System.currentTimeMillis();
LOGGER.info("CompiledFilter execute time: {} ms", (etm-stm));
}
catch (Throwable ex) {
LOGGER.error(ex.getMessage(), ex);
System.exit(1);
}
}
private static String DynFilterEvenGt500() {
return
"""
package com.polarsparc.jdk.compiler;
import com.polarsparc.jdk.compiler.Filter;
public class DynFilterEvenGt500 implements Filter {
@Override
public boolean filter(int num) {
if (num > 500 && (num % 2) == 0) {
return true;
}
return false;
}
}
""";
}
private static String DynFilterOddLt500() {
return
"""
package com.polarsparc.jdk.compiler;
import com.polarsparc.jdk.compiler.Filter;
public class DynFilterOddLt500 implements Filter {
@Override
public boolean filter(int num) {
if (num < 500 && (num % 2) != 0) {
return true;
}
return false;
}
}
""";
}
}
If the user chooses DynFilterEvenGt500, an even Filter implementation is generated and executed. On the other hand, if the user chooses DynFilterOddLt500, an odd Filter implementation is generated.
To execute the code from Listing.13 for the option DynFilterEvenGt500, open a terminal window and run the following commands:
$ cd $HOME/java/JavaCompiler
$ mvn exec:java -Dexec.mainClass="com.polarsparc.jdk.compiler.CompiledFilter" -Dexec.args="DynFilterEvenGt500"
The following would be the typical output:
[INFO] Scanning for projects... [INFO] [INFO] --------------< com.polarsparc.javacompiler:JavaCompiler >-------------- [INFO] Building JavaCompiler 1.0 [INFO] from pom.xml [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- exec:3.5.0:java (default-cli) @ JavaCompiler --- 2025-01-19 15:12:48:885 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.DynamicCompiler - Compilation successful!!! 2025-01-19 15:12:48:886 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.ByteArrayClassLoader - Method findClass() called for class com.polarsparc.jdk.compiler.DynFilterEvenGt500 2025-01-19 15:12:48:886 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 846 2025-01-19 15:12:48:886 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 808 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 512 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 690 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 626 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 640 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 792 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 890 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 906 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 634 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 940 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 562 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 828 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 622 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 878 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 898 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 514 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 898 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 890 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 892 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter: 998 2025-01-19 15:12:48:887 [com.polarsparc.jdk.compiler.CompiledFilter.main()] INFO com.polarsparc.jdk.compiler.CompiledFilter - CompiledFilter execute time: 1 ms [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 0.474 s [INFO] Finished at: 2025-01-19T15:12:48-05:00 [INFO] ------------------------------------------------------------------------
With this, we conclude the hands-on coding and demonstrations for this article - let the Force be with you Luke !!!
References