I recently discovered that, under certain circumstances, synchronisation mechanisms offered by java.util.concurrent leak memory. When making a timed wait against, say, CountDownLatch, the latch will allocate a small object that is not garbage collected until the latch itself is garbage collected.
Our application has several CountDownLatches which last the lifetime of the application, and, in certain cirucmstances, the application uses a timed acquire against them. Unfortunately those certain circumstances appeared in production, and persisted for much longer than they do in test. Eventually the application ran out of memory, but not before starting to run very slowly, which increased lock contention, causing truly bizarre behaviour well before the application started to throw OutOfMemoryErrors.
The root of the problem is a bug in AbstractQueuedSynchronizer, and since AbstractQueuedSynchronizer is used to implement other synchronizer mechanisms provided by java.util.concurrent, ReentrantLock, Semaphore, and CountDownLatch, amongst others, all exhibit the bug.
Sun have known about this problem since March 2005, and they have made a decision not to fix it. I can understand this decision – any bug introduced by the fix would have wide-ranging consequences. I don’t agree with it, but I understand it. However, would it have killed Sun to stick a little note about the bug into the Javadoc?
At this point, I am put in mind of The Hitchhiker’s Guide:
PROSSER: But Mr. Dent, the plans have been available in the local planning office for the last nine months!
ARTHUR: Yes, well, as soon as I heard, I went straight round to see them. You hadn’t gone out of your way to call attention to them, had you? I mean, like actually telling anybody.
PROSSER: The plans were on display -
ARTHUR: On display? I had to go down to the cellar to find them!
PROSSER: That’s the display department!
ARTHUR: With a flashlight.
PROSSER: The lights had probably gone out.
ARTHUR: So had the stairs.
PROSSER: But you found the notice, didn’t you?
ARTHUR: Yes, I did. It was “on display” in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying, “Beware of the Leopard.”
Hmmm…. looks like Arthur has a good point, but then Sun Prosser says:
PROSSER: Mr. Dent, have you any idea how much damage that bulldozer would suffer if I just let it roll straight over you?
ARTHUR: How much?
PROSSER: None at all.
I had a look at the Google Maps Javascript API. It put me in mind of the Melbourne Metropolitan Fire Brigade’s active incident display – recently developed by my colleague, Chris – so I wrote a quick Python script to scrape the MFB‘s page and jammed the data onto Google Maps.
I ended up with this prototype.
I’ll show the MFB, but I’m not sure that Google Maps is entirely appropriate for them, since Google reserve the right to show advertising on their maps.
PS: There are often quiet times with no incidents. If you aren’t seeing any, try again in 20 mintures.
Sun have recently released Project Blackbox – 250 T1000 servers in a box.
I Want One. Or Two. Perhaps if I put it on my Amazon wishlist…
For those of us with weird Java threading problems, StackTrace is magic. Try this:
I heard the Hot Air Balloon, Manager and Engineer joke several years ago, and it’s still funny. Lots of other people seem to think so too.
As previously threatened, I’m rewriting my weblogging software again, in Django. Source code is up on Google Code.
The CGSociety is holding the EON Challenge, a CG art competition revolving around Greg Bear’s novel, EON. Can hardly wait to see the results! (via Malcolm)
Inspired by today’s Daily WTF, here’s the Google search for evil genius code. That search led me to a file named LargeCollectionOfVirusSources_virus00.zip, which seems to show that virus authors comment their code quite well. Digg ing a little on the archive’s site, I found a neat page of hacking tutorials.
In other news, Andy pointed me in the direction of Oracle to_char conversions, a great source of Daily WTF candidates, including this:
select /*+ use_nl(w,s) */
... a whole bunch of stuff ...
from v$session_wait w,v$session s
where w.sid = s.sid and (to_char(w.sid) = '0' or '0' = '0')
Google code search found Cardboard Schedule, which is tucked away in a zip file in the downloads section of my site. Neat. (via Ned Batchelder, who points to Jason Kottke’s list of Google code search hacks.)
Philip Greenspun writes, “The precision of modern avionics may well have contributed to” the recent 737 vs. business jet crash in Brazil. He goes on to argue that having a bit of randomness in the system makes it far less likely that two planes will end up in the same place at the same time.
This eye-witness account of the crash is also worth a read.
It’s great to see that the PSF Infrastructure committee have (almost) chosen an issue tracker. What’s disappointing, though, is the reaction over on comp.lang.python.
Giovanni and friends should remember that this is the end of the process for choosing a tracker, not the start. People that care deeply about Python development, as they appear to do, should have been reading python-dev in April June, July, or August. Many people contributed to these discussions. Why didn’t you?
The thing that really gets my goat, though, is this kind of polemic, which unfairly casts FUD on the motives, intelligence and judgement of the individuals that sit on the Infrastructure committee, the PSF generally, and the JIRA development team in particular.
Maybe I’m just annoyed because I’m seeing people I respect – including friends who work at Atlassian – being beat up for no good reason. Whatever. It’s offensive. Please stop.
So I walked over to David’s desk, and tried to hand him a dollar coin. He didn’t take it. He told me he’d rather I gave the dollar to iStockphoto, so they could send him 20 cents.
Apparently there’s more to this photography gig than money.
Sometimes software is cheaper if you buy from a distributor. For example, ESRI‘s MapObjects for Java costs $5000, if you buy direct, or just $50 if you go to the Cheap Software Megastore.
Update: Apparently I don’t do irony all that well. Let’s be clear here: I’m not advocating that you rush out to the Cheap Software Megastore and buy ESRI‘s software illegally. I’d be surprised if the warez shop sold any copies of MapObjects, since there’s not a lot of applications where you wouldn’t also want a great big ESRI server behind it, and the server software costs a lot more than $5K. In summary, don’t smoke, wear a seat belt and never, ever buy warez.
I seem to have been flat out these last few months, and it’s time to re-evaluate and re-prioritise my list of hobby projects. The ones on my mind are:
Hmmm. Looking at this list, I’d say the Django blog rewrite is next.
I was quite enjoying the new Google Reader for a while there. Then it started showing me items I’d already read. Then it started miscalculating the counts on the left hand side. Now it tells me that I am not subscribed to any feeds at all. It’s not normal for Google to just forget stuff is it?
Update: I was being a doofus. See below.
Actually, I think I have an explanation. I blame Simon. I reckon the sudden shift in Internet traffic patterns, caused by billions around the world wanting to check out the new item on Simon’s RSS feed, broke Google.
Update: Just figured it out. It’s quite interesting, in a way. It turns out I had two Google Accounts under different email addresses, and I was signed into Gmail with one account, and Google Reader with the other. I had just assumed they shared a login session, but obviously not. Just to be sure, I deleted the second account, and I am relieved – beyond my ability to express to any non-geek – to learn Google hadn’t lost my feed list.