Java on a 1000 Cores - Tales of Hardware / Software CoDesign

> What I can imagine is a purpose-built CPU that would make the JIT's job a lot easier and faster than compiling for x86 or ARM. Such a machine wouldn't execute raw Java bytecode, rather, something a tiny bit more low-level. This is approximately exactly what Azul Systems did, doing a bog-standard RISC with hardware GC barriers and transactional memory. Cliff Click gave an excellent talk on it [0] and makes your argument around 20:14. [0] ▶ this video

Note that "short lived objects" and "stack allocated objects" aren't exactly the same set. Common Lisp has a dynamic-extent declaration, which can be used to stack allocate [1]. Go and Java implementations perform escape analysis. There can have some wins if stack allocatability is tracked at runtime, rather than compile-time [2]; Cliff Click reported roughly doubling the number of stack-allocated objects by using escape detection rather than analysis [3]. At least the Inkscape vector editor uses the Boehm garbage collector [4]. There are several programming language implementations which use Boehm and C, but one wonders if it is still a "C codebase" then. [1] link [2] Henry Baker, CONS should not CONS its arguments link [3] ▶ this video [4] link

Azul Systems (who today are mainly a JVM vendor) started by building large multicore boxes (Vega) to do Java workloads - there is an interesting tech talk about it: ▶ this video (2009) They built a "really good JIT target" rather than direct bytecode for reasons explained in the video. Curious to hear from anyone who ever used one...

There are some interesting challenging when scaling this large with any system. Peter referenced some of Azul System's stuff and their custom GC. Cliff Click from Azul has done some great Google Tech Talks about JVM development and scaling JVM's out to really big systems.[1][2] [1] A JVM Does That? - link [2] Java on a 1000 Cores - ▶ this video

I have not suggested that they put C4 or the orginal passless as a standard. While they did focuse on a perticular market segment the performance and paustime numbers they showed could easly have been usful for tons and tons of other markets. While C4 does currently require OS enhancements, they could make it without it, it would just be slower. However the problem that Azul has tried to overcome with the OS could have been much more powerful attacked with the full force of oracle/sun behind it. They once put some linux patches but it was never there focus and it never went upstream. At a big company this technogoly could have had a bigger impact, they could have talked to microsoft, apple and the linux team. I know that Azul also tried to strike up a conversation with Intel but in there words "it was like moving a mountain". And im not suggesting Sun/Oracle should have done this for the greater good of all humans. They would would have created a gigantig benefit for java products over the competition C# and C++ in the server space. I rember talks by Cliff Click that mentioned that they were really suprised that Sun did not have more intresst in them, considering some simple things that Sun could have to get huge increses in some of the high end server systems they were selling (am searching for the refernce ...). Edit: See Cliff Click talk about intel and sun. ▶ this video t=54m33s He mentions that the sun shareholders should put there exutive staff into prison.

I don't think that there's anything magical about C4. All the basics are discernible from this presentation: ▶ this video Azul originally started out as a hardware company. A pauseless collector is easy as long as you have transactional memory (at a minimum hardware which supports strong LL/SC, allowing you to build the rest in software). No commodity CPUs support this, then or now, so Azul's first product was a specialized CPU and memory architecture with real transactional memory semantics. But nobody wants to buy specialized hardware these days. So they ended up having to emulate transactional memory on x86. That means _emulating_ the memory architecture of their original CPU. Again, there's no secret sauce to do doing this. Things like QEMU are free--which is to say, many people know how to do it. It just takes a lot of programmer hours to build it, and many, many more to optimize it. Long story short, Azul's collector is truly pauseless, but its transactional throughput in the vast majority of workloads sucks because it's an emulator (for the JVM) built atop another emulator (for their original memory architecture). And while you can speed up emulating the JVM using JIT techniques, that doesn't work for emulating a transactional memory architecture. There are endless tweaks you can make, including playing around with memory page tables, etc, but at the end up the day you're still stuck emulating the semantics you need, causing huge latencies relative to your clock speed, whic

Probably this one: link Anything from Cliff Click is quite good. This guy has a very deep understanding of compilers, virtual-machines, CPUs, and their interactions. A JVM does that? link Java on a 1000 Cores ▶ this video

Here's an excellent google tech talk by Cliff Click of Azul talking about their hardware. Among the firehose of information in that talk, he mentions that the hardware guys wanted to add directly execute JVM bytecode. He told them not to bother since he could JIT it better and they wouldn't be able to support all the bytecodes anyway. I'd love to know more about why that is. ▶ this video

As Click discusses in his Google tech talk ( ▶ this video ) Intel could implement a hardware read-barrier and radically change the GC performance picture for commodity systems.