Tuesday, October 16, 2007
Monday, October 15, 2007
Java SecureRandom weirdness
Ok, now I really lied two posts ago since I'm going to go off topic again. I clearly have no ability to focus. I would have posted something within focus this weekend, but Half Life 2: Episode 2 came out and I really had no choice except to play it all weekend.
So anyways, for my own and perhaps others' benefit, I'm going to document my experience with debugging some Java weirdness because it was one of the more interesting problems I've had to deal with. So I'll dive in:
We have a server-side process that runs on a Linux server and processes a lot of oracle database data and converts it into an H2 database (details are unimportant). On one particular machine, this process would hang forever. There were no errors, no exceptions, nothing; it just hung. Last Friday I was tasked with figuring out why, and it being Friday I wasted a couple hours trying to debug what turned out to be a completely unrelated process (my brain was obviously fried). So fresh this morning I had another go at it. I had telnet access to the server, so I made use of jstack, a nifty little utility. Here's the important output:
- java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; information may be imprecise)- java.io.FileInputStream.read(byte[], int, int) @bci=4, line=199 (Compiled frame)- java.io.BufferedInputStream.fill () @bci=175, line=218 (Interpreted frame)- java.io.BufferedInputStream.read1(byte[], int, int) @bci=44, line=258 (Interpreted frame)- java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=317 (Interpreted frame) - sun.security.provider.SeedGenerator$URLSeedGenerator.getSeedByte() @bci=12, line=453 (Interpreted frame)- sun.security.provider.SeedGenerator.getSeedBytes(byte[]) @bci=11, line=123 (Interpreted frame)- sun.security.provider.SeedGenerator.generateSeed (byte[]) @bci=4, line=118 (Interpreted frame)- sun.security.provider.SecureRandom.engineGenerateSeed(int) @bci=5, line=114 (Interpreted frame)- sun.security.provider.SecureRandom.engineNextBytes(byte[]) @bci=40, line=171 (Interpreted frame) - java.security.SecureRandom.nextBytes(byte[]) @bci=5, line=433 (Interpreted frame)- java.security.SecureRandom.next(int) @bci=17, line=455 (Interpreted frame)- java.util.Random.nextLong() @bci=3, line=284 (Interpreted frame)
The thread was stuck trying to read a file as a randomness source. After some digging, here's what I found. Basically class SecureRandom is reading a file that generates random bits so it can create a pseudo-random number. As the forum folks found, this file is specified in $JAVA_HOME/jre/lib/security/java.security under securerandom.source, and the default is /dev/random. Unfortunately reading /dev/random will block until the OS has entropy data (from typing on the keyboard or other somewhat random happenings in the computer). Here's some more detail.
So if you don't need really random randomness you can use /dev/urandom which will not block. That sounds easy enough, but when I checked this in java.security, it was already pointing at /dev/urandom. So what the heck was going on?
Apparently the H2 driver that we are using calls SecureRandom.getInstance("SHA1PRNG") to create the SecureRandom class. Inexplicably, this will end up using /dev/random no matter what the java.security file says. This bug report claims this is as designed. I'm sure there is a reason for this strangeness, but there is no explanation.
Here's a test program that shows the difference (pardon the lack of formatting):
import java.security.SecureRandom;
public class SeedTest {
public static void main(String args[]) {
try{
System.setProperty("java.security.debug","all");
System.out.println("I will try to instantiate SecureRandom now");
SecureRandom rand = SecureRandom.getInstance("SHA1PRNG");
//SecureRandom rand = new SecureRandom(); this makes all the difference
int dummy = rand.nextInt();
System.out.println (dummy);
System.out.println("FINISHED");
} catch (Exception e) {
e.printStackTrace();
}
}
Running this on the particular problematic machine hung with getInstance, but worked fine with new SecureRandom(). Yowza. After all this I'm not sure whose problem this is. Obviously the machine with /dev/random not working is at fault, but why is SecureRandom using /dev/random when I specifically tell it to use /dev/urandom? No idea.
So anyways, for my own and perhaps others' benefit, I'm going to document my experience with debugging some Java weirdness because it was one of the more interesting problems I've had to deal with. So I'll dive in:
We have a server-side process that runs on a Linux server and processes a lot of oracle database data and converts it into an H2 database (details are unimportant). On one particular machine, this process would hang forever. There were no errors, no exceptions, nothing; it just hung. Last Friday I was tasked with figuring out why, and it being Friday I wasted a couple hours trying to debug what turned out to be a completely unrelated process (my brain was obviously fried). So fresh this morning I had another go at it. I had telnet access to the server, so I made use of jstack, a nifty little utility. Here's the important output:
- java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; information may be imprecise)- java.io.FileInputStream.read(byte[], int, int) @bci=4, line=199 (Compiled frame)- java.io.BufferedInputStream.fill () @bci=175, line=218 (Interpreted frame)- java.io.BufferedInputStream.read1(byte[], int, int) @bci=44, line=258 (Interpreted frame)- java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=317 (Interpreted frame) - sun.security.provider.SeedGenerator$URLSeedGenerator.getSeedByte() @bci=12, line=453 (Interpreted frame)- sun.security.provider.SeedGenerator.getSeedBytes(byte[]) @bci=11, line=123 (Interpreted frame)- sun.security.provider.SeedGenerator.generateSeed (byte[]) @bci=4, line=118 (Interpreted frame)- sun.security.provider.SecureRandom.engineGenerateSeed(int) @bci=5, line=114 (Interpreted frame)- sun.security.provider.SecureRandom.engineNextBytes(byte[]) @bci=40, line=171 (Interpreted frame) - java.security.SecureRandom.nextBytes(byte[]) @bci=5, line=433 (Interpreted frame)- java.security.SecureRandom.next(int) @bci=17, line=455 (Interpreted frame)- java.util.Random.nextLong() @bci=3, line=284 (Interpreted frame)
The thread was stuck trying to read a file as a randomness source. After some digging, here's what I found. Basically class SecureRandom is reading a file that generates random bits so it can create a pseudo-random number. As the forum folks found, this file is specified in $JAVA_HOME/jre/lib/security/java.security under securerandom.source, and the default is /dev/random. Unfortunately reading /dev/random will block until the OS has entropy data (from typing on the keyboard or other somewhat random happenings in the computer). Here's some more detail.
So if you don't need really random randomness you can use /dev/urandom which will not block. That sounds easy enough, but when I checked this in java.security, it was already pointing at /dev/urandom. So what the heck was going on?
Apparently the H2 driver that we are using calls SecureRandom.getInstance("SHA1PRNG") to create the SecureRandom class. Inexplicably, this will end up using /dev/random no matter what the java.security file says. This bug report claims this is as designed. I'm sure there is a reason for this strangeness, but there is no explanation.
Here's a test program that shows the difference (pardon the lack of formatting):
import java.security.SecureRandom;
public class SeedTest {
public static void main(String args[]) {
try{
System.setProperty("java.security.debug","all");
System.out.println("I will try to instantiate SecureRandom now");
SecureRandom rand = SecureRandom.getInstance("SHA1PRNG");
//SecureRandom rand = new SecureRandom(); this makes all the difference
int dummy = rand.nextInt();
System.out.println (dummy);
System.out.println("FINISHED");
} catch (Exception e) {
e.printStackTrace();
}
}
Running this on the particular problematic machine hung with getInstance, but worked fine with new SecureRandom(). Yowza. After all this I'm not sure whose problem this is. Obviously the machine with /dev/random not working is at fault, but why is SecureRandom using /dev/random when I specifically tell it to use /dev/urandom? No idea.
Saturday, October 6, 2007
Thursday, October 4, 2007
I'm going back to 1995
This will be my first in series of posts on one subject. Yes, that's correct. I will attempt to maintain some focus and stay on one topic instead of posting about whatever happens to be going on in my brain at the moment.
I have entered into the world of 1995 with the Win32 api and visual c++ for no reason other than to waste my extremely valuable time. This has been the site of my realization-that-everyone-else-realized-about-fifteen-years-ago. It dawned on me that despite the quirks and the crap throughout Windows, programming in it takes a lot of the guesswork out of things. For instance, if I were to make an application for Linux, there are many differences among the different distributions (filesystem layout comes to mind first) that I would have to account for manually within my application. Another example: say I want to capture the video from a webcam in my application. With Windows there is one function I can call, check for error, and go from there. With Linux, I have to make sure that libraries to handle low-level access to the webcam are installed properly and I have to know where they are for many distributions. Most likely I want to just support one api so I have to make sure it is installed. If I am writing an application with a wide variety of customers, it is easy to see why Windows is the easy choice here (ignoring the fact that it is already installed on 99% of personal computers to begin with). I guess I've seen the value of the Microsoft "platform," despite its many faults (and steep monetary cost).
I have entered into the world of 1995 with the Win32 api and visual c++ for no reason other than to waste my extremely valuable time. This has been the site of my realization-that-everyone-else-realized-about-fifteen-years-ago. It dawned on me that despite the quirks and the crap throughout Windows, programming in it takes a lot of the guesswork out of things. For instance, if I were to make an application for Linux, there are many differences among the different distributions (filesystem layout comes to mind first) that I would have to account for manually within my application. Another example: say I want to capture the video from a webcam in my application. With Windows there is one function I can call, check for error, and go from there. With Linux, I have to make sure that libraries to handle low-level access to the webcam are installed properly and I have to know where they are for many distributions. Most likely I want to just support one api so I have to make sure it is installed. If I am writing an application with a wide variety of customers, it is easy to see why Windows is the easy choice here (ignoring the fact that it is already installed on 99% of personal computers to begin with). I guess I've seen the value of the Microsoft "platform," despite its many faults (and steep monetary cost).
Saturday, September 29, 2007
"Clinton said such an account program would help Americans get back to the tradition of savings that she remembers as a child, and has become harder to accomplish in the face of rising college and housing costs."
Yes, the good old America tradition of not saving your own money, but the government's money (which is, in essence, every taxpayer's money minus 50% because of government waste). What a brilliant idea. Now I wonder, maybe people should perhaps, this will sound crazy I know, save their OWN money? Clinton's response: "She argued that wealthy people get to have all kinds of tax incentives to save, but most people can't afford to do that.'" Well, this is going to even more crazy, but I wonder if the reason people can't afford to save is that they are blowing all their money on things that can't afford? Now I'm just going to jump off the deep end, so bear with me, but what if people lived within their means, didn't buy twice as much house as they can afford, didn't buy the latest and greatest car, and shudder, saved some money for the future? Whoaaaaa, that's just crazy talk.
Yes, the good old America tradition of not saving your own money, but the government's money (which is, in essence, every taxpayer's money minus 50% because of government waste). What a brilliant idea. Now I wonder, maybe people should perhaps, this will sound crazy I know, save their OWN money? Clinton's response: "She argued that wealthy people get to have all kinds of tax incentives to save, but most people can't afford to do that.'" Well, this is going to even more crazy, but I wonder if the reason people can't afford to save is that they are blowing all their money on things that can't afford? Now I'm just going to jump off the deep end, so bear with me, but what if people lived within their means, didn't buy twice as much house as they can afford, didn't buy the latest and greatest car, and shudder, saved some money for the future? Whoaaaaa, that's just crazy talk.
Friday, September 28, 2007
Thursday, September 27, 2007
There's a reason why I chose the science/engineering/technology route for my life. In technology progress in inevitable; there may be periods when advancement is slower than it could be, but there is always a guarantee that it moves forward. I can guarantee you that in five years computers will be faster, phones will be smaller, materials will be lighter and stronger, and batteries will last longer. It feels good to know that we will always be moving forward.
As opposed to governments, which remain eternally stupid, and are guaranteed to make the same mistakes over, and over, and over, and over again. I'm not going to go into specifics because there are too many to mention. I'm sure you can think of a few.
In the meantime, one of my favorite quotes these days is from Benjamin Franklin:
"Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety."
As opposed to governments, which remain eternally stupid, and are guaranteed to make the same mistakes over, and over, and over, and over again. I'm not going to go into specifics because there are too many to mention. I'm sure you can think of a few.
In the meantime, one of my favorite quotes these days is from Benjamin Franklin:
"Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety."
Subscribe to:
Posts (Atom)