Apache Derby Won't Start Anymore!

We use Apache Derby a lot as a stand-alone SQL database for software development courses. After Java 7 came out we started to have problems with Derby on computers in the classroom. Sometimes it would run, sometimes it wouldn't.

Sometimes it would run a computer with Java 7 or even 8, other times it would not. The common denominator was Java 7+, and while early releases of Java 7 do not exhibit any issues it is just easier to lump everything from Java 7 onward together. So when would it run? If the computer also had Java 6, and JAVA_HOME was set to that JRE, and if Derby was launched with a script that checked JAVA_HOME.

The real problem of course is that Derby is a Java application and the later JREs are locked down more and throw a security fit when Derby wants to bind to a TCP/IP port.

So the simple solution is to just write a script to wrap Derby and use Java 6. But, you may not have Java 6. In fact, you should not have it, there are simply too many security issues for it to be on your computer. Oh bother, what to do now?

Option 1: Find the JRE and configure the global security policy

This is fastest way to solve the security issue, but it does affect every other application using the JRE. And locating the correct JRE if you have multiples is a topic unto itself, this post explains how to do that: Which Java am I using? And finally, you have to have administrative privileges on the computer.

We can fix the security by editing the java.policy file for the JRE that is running. The java.policy file will be found in the JRE under the folder lib/security.

You will need to edit this file as an administrator or you will not be able to save your changes. On a Microsoft Windows computer launch Wordpad or another text editor as an administrator and then open the file. If you run the editor as an administrator first, then you can save the file. It will not work if you just edit the file, even if you have administrator privileges. I picked Wordpad because the end-of-line characters are not always Windows compliant and Notepad does not deal with them very well.

On an OS X or Linux computer use sudo to run an editor with root privileges and then open the file.

This file consists of "grant" blocks that grant privileges. Grant blocks typically have a constraint that defines what they apply to. Search for the grant block without a constraint, that one applies to all running JREs.

grant {

Add the following line to the top of the grant block. This will allow Derby rights to bind to the TCP/IP port that it listens for connection on:

grant {
@tab;permission java.net.SocketPermission "localhost:1527", "listen";

And that is is. Save the file and launch Derby. You should be successful.

Option 2: Using a local security policy

The other, and maybe better, solution is to launch Derby using its own policy file. No administrative privileges required. This file is based on the template that is provided with Derby, but we fixed a couple of problems with it.

Working on the assumption that the script to launch Derby will be in the Derby folder, I changed the codeBase of the policies to relative paths to the DLL files. The original script has ${derby.install.url} and we are instructed to replace that with a hardwired path. I chose to change that to ${derby.system.home} and then set that property before Java is launched. The reason is that it appears multiple times in the file, and the property ${derby.dbca.traceDirectory} which is also in the file defaults to home when it is not set. I also limited the file access to the home directory and below.

The only other change is to the startup script, where I define the home directory as the current directory and specify the policy file the application will be launched under:

java -Dderby.system.home=. -Djava.security.manager -Djava.security.policy=derby.policy org.apache.derby.drda.NetworkServerControl start

Here is the complete text of my version of the derby.policy file. The changes that I made are in bold text:

// derby.policy
//
// This policy file is derived from the template provided with Derby 10.11. It has been altered
// to use codeBase paths relative to the installation directory. This supports double-clicking
// on the command file in Windows or running the scripts from the directory on OS X and Linux.
//
// The permissions are divided for the specific requirements of the main database code in the
// derby.jar file and the networking code in derbynet.jar.
//

//
// Licensed to the Apache Software Foundation (ASF) under one or more
// contributor license agreements. See the NOTICE file distributed with
// this work for additional information regarding copyright ownership.
// The ASF licenses this file to You under the Apache License, Version 2.0
// (the "License"); you may not use this file except in compliance with
// the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//

// This template policy file gives examples of how to configure the
// permissions needed to run a Derby network server with the Java
// Security manager.
//
grant codeBase "file:${derby.system.home}/lib/derby.jar"
{
@tab;// These permissions are needed for everyday, embedded Derby usage.
@tab;//
@tab;permission java.lang.RuntimePermission "createClassLoader";
@tab;permission java.util.PropertyPermission "derby.*", "read";
@tab;permission java.util.PropertyPermission "user.dir", "read";

@tab;// The next two properties are used to determine if the VM is 32 or 64 bit.
@tab;//
@tab;permission java.util.PropertyPermission "sun.arch.data.model", "read";
@tab;permission java.util.PropertyPermission "os.arch", "read";
@tab;permission java.io.FilePermission "${derby.system.home}","read";
@tab;permission java.io.FilePermission "${derby.system.home}${/}-", "read,write,delete";

@tab;// This permission lets a DBA reload the policy file while the server is
@tab;// still running. The policy file is reloaded by invoking the
@tab;// SYSCS_UTIL.SYSCS_RELOAD_SECURITY_POLICY() system procedure.
@tab;//
@tab;permission java.security.SecurityPermission "getPolicy";

@tab;// This permission lets you backup and restore databases to and from
@tab;// arbitrary locations in your file system.
@tab;//
@tab;// This permission also lets you import/export data to and from arbitrary
@tab;// locations in your file system.
@tab;//
@tab;// You may want to restrict this access to specific directories.
@tab;//
@tab;// permission java.io.FilePermission "<<ALL FILES>>", "read,write,delete";

@tab;// Permissions needed for JMX based management and monitoring.
@tab;//
@tab;// Allows this code to create an MBeanServer:
@tab;//
@tab;permission javax.management.MBeanServerPermission "createMBeanServer";

@tab;// Allows access to Derby's built-in MBeans, within the domain
@tab;// org.apache.derby. Derby must be allowed to register and unregister these
@tab;// MBeans. It is possible to allow access only to specific MBeans,
@tab;// attributes or operations. To fine tune this permission, see the javadoc of
@tab;// javax.management.MBeanPermission or the JMX Instrumentation and Agent
@tab;// Specification.
@tab;//
@tab;permission javax.management.MBeanPermission "org.apache.derby.*#[org.apache.derby:*]", "registerMBean,unregisterMBean";

@tab;// Trusts Derby code to be a source of MBeans and to register these in the
@tab;// MBean server.
@tab;//
@tab;permission javax.management.MBeanTrustPermission "register";

@tab;// getProtectionDomain is an optional permission needed for printing
@tab;// classpath information to derby.log
@tab;//
@tab;permission java.lang.RuntimePermission "getProtectionDomain";

@tab;//
@tab;// The following permission must be granted for Connection.abort(Executor) to
@tab;// work. Note that this permission must also be granted to outer
@tab;// (application) code domains.
@tab;//
@tab;permission java.sql.SQLPermission "callAbort";

@tab;// Needed by file permissions restriction system:
@tab;//
@tab;permission java.lang.RuntimePermission "accessUserInformation";
@tab;permission java.lang.RuntimePermission "getFileStoreAttributes";
};


grant codeBase "file:./lib/derbynet.jar"
{
@tab;// These permissions lets the Network Server manage connections from clients.

@tab;// Accept connections from any host. Derby is listening to the host interface
@tab;// specified via the -h option to "NetworkServerControl start" on the command
@tab;// line, via the address parameter to the
@tab;// org.apache.derby.drda.NetworkServerControl constructor in the API or via
@tab;// the property derby.drda.host; the default is localhost. You may want to
@tab;// restrict allowed hosts, e.g. to hosts in a specific subdomain,
@tab;// e.g. "*.example.com".
@tab;permission java.net.SocketPermission "*", "accept";

@tab;// Allow the server to listen to the socket on the default port (1527).
@tab;// If you have specified another port number with the -p option to
@tab;// "NetworkServerControl start" on the command line, or with the portNumber
@tab;// parameter to the NetworkServerControl constructor in the API, or with the
@tab;// property derby.drda.portNumber, you should change the port number in the
@tab;// permission statement accordingly.
@tab;permission java.net.SocketPermission "localhost:1527", "listen";

@tab;// Needed for server tracing.
@tab;//
@tab;permission java.io.FilePermission "${derby.drda.traceDirectory}${/}-", "read,write,delete";

@tab;// Needed by file permissions restriction system:
@tab;//
@tab;permission java.lang.RuntimePermission "accessUserInformation";
@tab;permission java.lang.RuntimePermission "getFileStoreAttributes";
@tab;permission java.util.PropertyPermission "derby.__serverStartedFromCmdLine", "read, write";

@tab;// JMX: Uncomment this permission to allow the ping operation of the
@tab;// NetworkServerMBean to connect to the Network Server.
@tab;//
@tab;// permission java.net.SocketPermission "*", "connect,resolve";

@tab;// Needed by sysinfo. The file permission is needed to check the existence of
@tab;// jars on the classpath. You can limit this permission to just the locations
@tab;// which hold your jar files.
@tab;//
@tab;// In this template file, this block of permissions is granted to
@tab;// derbynet.jar under the assumption that derbynet.jar is the first jar file
@tab;// in your classpath which contains the sysinfo classes. If that is not the
@tab;// case, then you will want to grant this block of permissions to the first
@tab;// jar file in your classpath which contains the sysinfo classes. Those
@tab;// classes are bundled into the following Derby jar files:
@tab;//
@tab;// derbynet.jar
@tab;// derby.jar
@tab;// derbyclient.jar
@tab;// derbytools.jar
@tab;//
@tab;permission java.util.PropertyPermission "user.*", "read";
@tab;permission java.util.PropertyPermission "java.home", "read";
@tab;permission java.util.PropertyPermission "java.class.path", "read";
@tab;permission java.util.PropertyPermission "java.runtime.version", "read";
@tab;permission java.util.PropertyPermission "java.fullversion", "read";
@tab;permission java.lang.RuntimePermission "getProtectionDomain";
@tab;//permission java.io.FilePermission "<<ALL FILES>>", "read";
};