Monday, October 15, 2007

Java SecureRandom weirdness

Ok, now I really lied two posts ago since I'm going to go off topic again. I clearly have no ability to focus. I would have posted something within focus this weekend, but Half Life 2: Episode 2 came out and I really had no choice except to play it all weekend.
So anyways, for my own and perhaps others' benefit, I'm going to document my experience with debugging some Java weirdness because it was one of the more interesting problems I've had to deal with. So I'll dive in:

We have a server-side process that runs on a Linux server and processes a lot of oracle database data and converts it into an H2 database (details are unimportant). On one particular machine, this process would hang forever. There were no errors, no exceptions, nothing; it just hung. Last Friday I was tasked with figuring out why, and it being Friday I wasted a couple hours trying to debug what turned out to be a completely unrelated process (my brain was obviously fried). So fresh this morning I had another go at it. I had telnet access to the server, so I made use of jstack, a nifty little utility. Here's the important output:

- java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; information may be imprecise)- java.io.FileInputStream.read(byte[], int, int) @bci=4, line=199 (Compiled frame)- java.io.BufferedInputStream.fill () @bci=175, line=218 (Interpreted frame)- java.io.BufferedInputStream.read1(byte[], int, int) @bci=44, line=258 (Interpreted frame)- java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=317 (Interpreted frame) - sun.security.provider.SeedGenerator$URLSeedGenerator.getSeedByte() @bci=12, line=453 (Interpreted frame)- sun.security.provider.SeedGenerator.getSeedBytes(byte[]) @bci=11, line=123 (Interpreted frame)- sun.security.provider.SeedGenerator.generateSeed (byte[]) @bci=4, line=118 (Interpreted frame)- sun.security.provider.SecureRandom.engineGenerateSeed(int) @bci=5, line=114 (Interpreted frame)- sun.security.provider.SecureRandom.engineNextBytes(byte[]) @bci=40, line=171 (Interpreted frame) - java.security.SecureRandom.nextBytes(byte[]) @bci=5, line=433 (Interpreted frame)- java.security.SecureRandom.next(int) @bci=17, line=455 (Interpreted frame)- java.util.Random.nextLong() @bci=3, line=284 (Interpreted frame)

The thread was stuck trying to read a file as a randomness source. After some digging, here's what I found. Basically class SecureRandom is reading a file that generates random bits so it can create a pseudo-random number. As the forum folks found, this file is specified in $JAVA_HOME/jre/lib/security/java.security under securerandom.source, and the default is /dev/random. Unfortunately reading /dev/random will block until the OS has entropy data (from typing on the keyboard or other somewhat random happenings in the computer). Here's some more detail.

So if you don't need really random randomness you can use /dev/urandom which will not block. That sounds easy enough, but when I checked this in java.security, it was already pointing at /dev/urandom. So what the heck was going on?

Apparently the H2 driver that we are using calls SecureRandom.getInstance("SHA1PRNG") to create the SecureRandom class. Inexplicably, this will end up using /dev/random no matter what the java.security file says. This bug report claims this is as designed. I'm sure there is a reason for this strangeness, but there is no explanation.

Here's a test program that shows the difference (pardon the lack of formatting):

import java.security.SecureRandom;

public class SeedTest {
public static void main(String args[]) {
try{
System.setProperty("java.security.debug","all");
System.out.println("I will try to instantiate SecureRandom now");
SecureRandom rand = SecureRandom.getInstance("SHA1PRNG");
//SecureRandom rand = new SecureRandom(); this makes all the difference
int dummy = rand.nextInt();
System.out.println (dummy);
System.out.println("FINISHED");
} catch (Exception e) {
e.printStackTrace();
}
}

Running this on the particular problematic machine hung with getInstance, but worked fine with new SecureRandom(). Yowza. After all this I'm not sure whose problem this is. Obviously the machine with /dev/random not working is at fault, but why is SecureRandom using /dev/random when I specifically tell it to use /dev/urandom? No idea.

3 comments:

Caleb said...

hey what the heck man you said you played half-life 2 Episode 2 all weekend and you post about debugging java? not all of us can use our weekend to play Episode 2 so we have to live through others like your self... I expect you to remedy the situation with your next post :-) *if at all possible

none said...

the java is fascinating. i don't understand it, so uh, try stories about video games.

Joseph Hitt said...

check these out:

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6521844

http://www.mail-archive.com/classpath-patches@gnu.org/msg08494.html

http://stackoverflow.com/questions/137212/how-to-solve-performance-problem-with-java-securerandom