My Blog

JUDCON 2010 - Berlin
----------------------------
Hosted at the Radial System V in east berlin, JUDCon spanned 2 cool autumn days at the start of October 2010.

I've written up a bunch of technical aspects I took from the presentations, occasionally sprinkled with my opinions and perspective. Inaccuracies on my part, differences in view or feedback/opinion are welcomed as comments.

DAY 1

----------------------------

There were 2 strands throughout day 1; Soa and Integration in one room, Cloud and Large Scale Distributed Systems in another.

Here's the agenda http://www.jboss.org/events/JUDCon/JUDCon2010Berlin/agenda.html

Having had a skim of the summaries I plumped for what I came to think of as the Infinispan (formerly Cloud) strand which ran across 4 talks from first thing in the morning into the middle of the afternoon, after which I planned to hop into the Integration room for the plugins presentation, but ended up sticking around for the integration testing talk.

My angle on this decision was that if some of the lead devs could give me insight into an upcoming, popular and powerful caching technology then this was worth more to me than discrete chunks on things like transfering state via xml (Sorry REST crowd!).

Session 1
Galder Zamarreño
Infinispan servers: beyond peer to peer data grids
----------------------------
HashMaps: now with front doors.

Based on the predicate that memory is the new disc and disc is the new tape (as memory is so cheap and uptime so ubiquitous); Infinispan provides mechanisms for people to use clustered memory management across jvms. It is a data grid akin to memcached. I'm no memcached expert but this talk stated that memcached cannot be clustered, the primary issue for service provision then becoming resilience. In English, if the box running it dies you're in trouble. As Infinispan is clustered you get this resilience amongst other things. This talk served as a good introduction to the subsequent 3 talks of the day.

As a datagrid Infinispan gives us things like data replication across nodes in the grid. We encounter problems communicating with non jvm environments. We can get around this, typically, by using a client server model over tcp.

The problem with Peer to Peer data replication is elasticity - complete replication (state transfer) can be slow to new nodes in the cluster. Apparently the client/server model can solve this as the server can point to new nodes as they're brought online - I'm not sure about this statement, I get the point of a single entry point giving load balance behaviour and allowing us to point to new nodes when they're ready but it doesn't make it quicker to move key/pair values around! Perhaps there is more to this than I took on board.

Infinispan can also give us per application data grids (more than one application in a jvm, across multiple jvms, where each app has it's own discrete datagrid and they communicate/replicate in a p2p fashion). Galdar's slides show this well. In this context a client server model:

can help as it can give 'per grid' security (one front door)
can allow you to deploy individual application upgrades and point to newer application instances as they come online, keeping the old ones until no longer needed
can allow per-jvm tuning requirements - if we need seperation then we can put things into a different jvm with separate garbage collection (tuning) requirements, for example one jvm may handle lots of web requests which would GC in one manor, we could then have datagrid jvms tuned for caching.

There are 4 server models they've built. There are trade-offs and benefits to each type.

Rest

http access: GET->retrieve & POST/PUT->store. +implementation agnostic (want python, php, ruby comms, etc).
need a servlet container.
can implement security in the web connector!

memcached (traditional LAMP stack http access cache)

not clustered (as stated before, if box dies then data dies as no replication),
static list of servers is required (config is therefore a bit of a dog)
Infinispan 'talks' memcached -> it uses memcached's protocol with Infinispan's back-end. This gives us our clustering (and thus redundancy) and adding new instances is easy.
A problem can appear with the fact that your data is (in one configuration of Infinispan's caching) only pushed to a subset of the nodes in the cluster. This can introduce roundtrips in a client server model (which is always going to have the same node entry point into the cluster, hence this node is always going to need to ask for the data from the node that holds it), but this is a protocol problem -> there's no way to easily solve this.

HotRod - Infinispan's new protocol to get around the limitations of memcached

gives us smart routing to the right server plus smart dynamic configuration.
'better' performance.
stores keys in byte array payloads to alow language agnostic interoperability (based on an apache project - I forget it's name)
currently a prototype
use case: long distance datagrid replication for backup / redundancy / discovery. eg linking data centers, etc. client server model -> hot standby site.

WebSocket

aimed at javascript usage - to pull data out of our cache into local javascript for operations in the view tier

Essentially there are limitations and benefits to each type - choose based on the following...

If have java clients -> use hot rod where you can

If have firewall restrictions -> use rest for its http access and text based transfer

If run non java clients -> use rest / memcached

If you have JS needs (although I cannot really think of any) -> use WebSocket

Next followed a demo of HotRod. Galdar had 2 active Inifinispan cache nodes and one standby node - this could be hosted offsite/in a backup environment. He consecutively killed the active nodes and we witnessed the data active on the standy node. Bringing the first active node back up showed and the data had been transferred back from the standby node.

We could wire HotRod up to a database for persistence - but it will persist to binary data that is proprietary to Infinispan. There may be a hibernate project in the works for this storage. Or could spin your own. OR could use Infinispan as the 2nd level cache in hibernate - snazzy.

Main future focus of work for these guys is the HotRod protocol.

Session 2
Galder Zamarreño
Highly precise Scalable data eviction in Infinispan
----------------------------
LIRS > LRU

The thrust behind this talk was to demonstrate the eviction algorithm used in Infinispan cache; how their implementations have evolved and why.

We started off with the Java Collections Framework (ie Lists, Maps, Trees, Sets -> HashMaps etc). Obviously these types of collections cannot grow forever or they shall leak memory (there just won't be any more room! OutOfMemoryException ahoy...). We must manually evict items based on your custom rules which can be a is going to be a PITA as systems grow.

Jboss Cache originally used an interceptor on puts() and gets() to the cache, then tied these to queues which were queried by an eviction thread to determine when to remove items. The problem they encountered in this scenario is that the eviction queues were filling up in busy environments. The suspected reason for this is the MVCC_blocking_read type used by the cache which means that non-blocking reads cause the eviction queue to fill....

In Infinispan 4.0.0 eviction used a linked list (in the form of a Collections doubly linked list - which gives rapid item removal, but slow iteration) so that eviction was less of a burden. This gets rid of the jboss cache issue.

I then missed an important bit as I was writing, when I came back the words "O(n) I.E. O log n operations not good enough, combined into a ConcurrentHashMap (doubly linked) -> ordered by creation time to get around this" were on the page. Apparently this gives good concurrent performance as pointers are allowed to be imperfect.

Regardless, apparently this implementation was fine all through testing; quick and stable. Until they hit R.C. 1 stage when it started hitting infinite loops. Apparently when tail operations on the queue were failing they were being retried again and again. They got around this by implementing queue operators using an iterator - the creation of which from a linked list is very slow. Hence when 4.0.0 went out the door it was compromised for speed.

For 4.1, the devs went back a step from their path and looked again at the Least Recently Used cache (LRU - http://en.wikipedia.org/wiki/Cache_algorithms#Least_Recently_Used) cache. The issue with which is that eviction is solely time based - it does not account for the number of times a cached item is actually being read from (hit). But it's still a nifty algorithm so we're not done with it in this story just yet....

Enter the LIRS algorithm. It is based on 2 papers written by [I apologies for I did not get your names, clever people] and enforces a form of inter-reference recency. In effect this means it is measuring how often each item is being used relative to other items in the cache (those around it), and from whom over a given time period. The end product of which is you get items that are being accessed consecutively and in a short space of time grouped together in the cache.

The actual implementation of this is pretty interesting; it involves the use of two queues and a smart algorithm for managing promotion. The 2 queues separately hold hot keys and recently accessed keys. Recently accessed keys may be pushed to the hot key queue if a number of conditions are met (but think hotter/newer than a hot key).

The upshot is one time cache accesses are evicted very quickly, whereas items accessed often or recently will be in the cache for longer and so will get hit! This is unlike the LRU where one time hits will not be evicted until they age off. The downside of this implementation is that it can require a lot of key switching which can lead to high contention (fighting for locks on the queues).

So close! What now? Enter: Batching eviction updates. So we move all updates to one place and perform the whole operation in a batch only when the queues hit predefined thresholds. "We let each thread deal with it's accesses". I'm not sure what this meant but it has big implications if each thread is managing it's own thresholds...

The end result of this is a Bounded, Concurrent Hash Map. It is highly concurrent, highly precise with implicit (barely noticiable) eviction.

LIRS is freely available to drop in to replace anybodies LRU as it is community software.

Conclusion was that in a large Berlin based company a deployment of LIRS had seen a 50-60% increase in throughput. Not bad.

The slides will be up for this that show the eviction algorithm in action - worth going through to see the slides describing the queue management.

Session 3

Continuous query in Infinispan
Mircea Markus

----------------------------

Good live demo

This was neat. It demo'd the combination of drools and Infinispan to implement complex event processing (CEP) and continuous queries (CQ).

CEP is where multiple events are always occuring in a 'complex' environment. The example was a stock trading platform which had real time, compund events we want to react to. If a stock falls below a certain threshold then sell, sell, sell!

Drools Live Queries (DLQ) give us our CQ. They're an approach to implementing CEP. They produce something akin to a 'materialised view' in a database. We cache the results of these queries in Infinispan. DLQ are marketed as more more powerful than straight up SQL for two reasons; they're more expressive (so you get more with less) and more readable. Also as we're operating on cached data from Infinispan we're quicker than SQL disk access. This last point feels a little funny to me - as I currently think the data must be being written somewhere by whatever is generating the information. I'd be surprised if it was bring written straight into the Infinispan cache and so it must be hitting the disk at some point to be pulled out by the CQs running in a timed operation.

DLQ's are defined by giving the query a name and initialising parameters in code. This format means was can reinstantiate them on the fly. We handle events generated when these queries are ran by assigning listeners - which are just pojos that extend a base class and optionally an interface. The interface gives you a replay() method which allows you to catch up on all pre-stored data that would generate events, without which you will only have the ability to generate events on future data - added(), removed() are called when items arrive/leave from the cached data - which might be fine for your query.

Infinispan's CQs are using ResultSetListeners on the DLQs to get their events. The results can be cached locally only - or pushed to other nodes in a cluster for more reads. A neat feature that Drools gives us is the ability to load these rules in at runtime - the real world benefit of which is that our trading platform would not have to be stopped and started whenever we wanted to add / remove system functionality. 100% uptime please - granted.

The CQ lifecycle goes: define -> deploy -> associate actions -> run -> disseminate results to nodes.

The live demo had a domain consisting of StockItems that were sold in a Trade (containing a quantity and a StockItem) which was performed by a Trader. We then saw a command line interface load a query from an xml file, then 2 other command line interfaces running in different processes could see and run the query. events would be generated for registered people in real time as the stock items were changed. Nice.

At the end someone pointed out all this is only any good if you're running replicated caching (all caches in the cluster contain the same data) rather than distributed (where data is copied around for redundancy but only to a subset of the whole cluster for speed). Mircea agreed and stated that this was currently an incubator Infinispan project and so the alternate mode was the next thing to work on.

Session 3

Transactions in Infinispan
Mircea Markus

----------------------------

Two phase caching

Hmm. Despite a tough post-lunch mid-afternoon slot, there were some good explanations on the uses of transactions in Infinispan out of the box, and your configuration options for transaction management. I took a lot from this and it revisited some fundamentals I'd consigned to the dusty shelves of my mind.

Effectively, for any data centric application like databases, transaction management is serious business. Management of your configuration will allow a trade off between maintaining consistency and application performance. Which may sound daft - as surely if your database is not looking after data integrity then it's not doing it's job - but this trade off is more angled at allowing things like dirty reads (out of synch data) across clustered locations for a period of time (until, say the next cache or db sync). Asynch protocol phases and lazy v eager locking are the main points discussed in this talk.

From the ground up, distributed transactions require a middleman to ensure both sides of the mechanism are holding up their end of the bargain. In your typical java ee container this is the job of the transaction manager (there are many implementations of this - jboss has it's own that it acquired called Arjuna). It's job is guarantee that 'all or nothing' will occur during a 2 phase commit. See ACID in rdbms if you want to know more about this.

For local writes, you don't need 2 phase behaviour. Even so, Infinispan has it's own transaction manger for local transactions which is still ACID compliant. Just use cache.getBatch(). It's locks on the data structure are write only by default, reads do not lock the system from writing at all.

Transaction cluster nodes and Lazy Locking

Effectively, when you write data to a node, it's only when you call commit() that it calls prepare() on associated nodes that contain the data. If this succeeds then you now hold a lock on those associated caches. commit() is then actually called to try to write the changes. Prepare() is the one with the high chance of failure out of the 2 calls. When the distributed commit() returns we commit locally and then return control to the user with the transaction closed and all data written successfully.

Lazy locking with Asynch commit

As with lazy locking above, but when commit() is called after the initial prepare() then we immediately return to the user. This is quicker as we do not wait for (or check) the result of the commit call - but there's a chance it didn't work and your data is out of sync!

Eager locking

Polar opposite of Asynch above. First you acquire locks on all the nodes you need. Hence they're froze in time waiting for you - no writes can occur (if configured then reads can still go ahead). Then prepare() is called (which actually doesn't do anything - you already have the locks!), the commit() phase is more efficient as it's effectively now just one call. But obviously this is more costly due to the time spent holding locks. 2 flavours - implicit (on each write) and explicit (on user demand).

But those famed deadlocks can occur if >1 thing is required to be locked in a transaction due to the order in which locks can be acquired. For example, if one transaction gets locks A,B and another gets locks B,A there's a chance we could be in trouble as both transactions would be waiting on each other's locks: classic deadlock (http://en.wikipedia.org/wiki/Deadlock#Examples). But this situation has to occur at exactly the same time. Work-arounds are to lexicographically order the keys (ensure that the transaction manager always acquires them in alphabetical order).

We can also use Infinispan's deadlock detection which will allow deadlocks to fail fast and at least one transaction will succeed. But this detection will come at a cost of cpu cycles and memory. Some graphs of cost verses the number of keys show wasn't too bad. Should use this when you have high key contention or if you are seeing many tx timeoutException rollbacks. Warned to measure first not just implement!

Transactions use MVCC (Multi Version Concurrency Control - optimised for reads) locks for writing.

Mircea mentioned they use an open source tool (which he has written) that monitors performance against cache implementations (including other products and their own releases) - CBF perhaps? Chase up his slides from jboss.org when they're available if you're interested

Session 5

Running a JBoss cluster in the clouds

Bela Ban

----------------------------

More than just multicast

Bela Ban was an excellent host.

His presentation was backed by a live demo throughout and focused on describing the 5 supported(ish) ways in which jboss clustering works under the hood. The aim being to smash the myth that jboss clustering doesn't work on EC2 - the issue being that out of the box jboss clustering uses ip multicasting and EC2 (and presumably other cloud providers - rackspace, gogrid, etc) don't support this mechanism between instances. The upshot is that members of a cluster cannot communicate, thus auto discovery fails so no clustered operations. Furthermore iptables -L and certain cisco devices block certain multicast addresses. Regardless, multicast is not always the way to go.

The crux of jboss clustering is JGroups. This is the layer in the stack which is responsible for making clustering so easy to use. It can be configured via a central file in server/all/deploy/cluster-service.xml Here you control the discovery mechanisms for cluster members.

The alternative clustering modes discussed are listed below.

static

Each instance has a hard coded list of nodes to perform clustering operations with. Conf: <TCPPING inital_hosts="..."

lookup service

Bela used a 'gossip router' to act as an intermediary for his nodes to talk to. When starting up they will attempt to register their hostname and ip and request the names of other instances. Mod_cluster was used in httpd for this functionality. Conf: <TCPGOSSIP ...

shared dir

can either be local if all on the same machine, or something like an NFS share. One folder per cluster, one file per node. can have a dir of dirs for mutiple clusters. <FILE_PING location="/your/path" ...

s3 bucket

a shared space on amazon ec2; but buckets are unique so must be named as such. apparently this will be the future default for EC@ jboss images and buckets can be auto generated <S3_PING location="..."

some_more_options_to_control_bucket_names_and_generation

database

not yet supported but planned to look at for the hackfest in the evening. simply use a table with rows for your instances. easy enough. See JGRP_1231.

The live demo showed usage of all of these pretty much effortlessly.

Something about http://www.jboss.org/stormgrind at the end briefly. If I had one criticism it was that it all seemed too easy and thus implicitly a bit lightweight - but then that's the hallmark of a good presentation.

Session 6

Seam in the clouds

Pete Muir

----------------------------

Roadmap for the church of the web-app

Stated aim was to build an integrated development platform which supports cloud based environments.

Sorry Pete, I don't use seam, but half the room seemed to and so I don't feel particuarly qualified to comment on how useful this talk may have been for those guys. It combined talking about CDI (context dependency injection - see jboss Weld) with JSF and repeated mention of the Java EE 6 spec. CDI is something Spring has had for a good while, but there are some neat features I saw (which may be in spring and I've just never used them...). One allowed you to easily create your own annotations that control how beans are deployed and handled (example had a user defined annotation dropped on a class definition that meant it was used it as an inifinispan cache setting it's caching properties and name - then there was a reference to an instance of that class which used the annotation on it - the container then wired everything up and the reference had direct cache access. which was nice.) Couple of other things; deltacloud (Ruby based local cloud instance), steamcannon (something to do with instant deployment to cloud environments) and a demo of jboss tools (which I do like - but had used before) that showed using these tools to control a seam 3 web app on a deployed cloud based machine.

Session 7 Frictionless Integration Testing in the cloud

----------------------------

Aliens in the clouds

I took a few things from this, but it was one of those talks which left me wanting more. There was a reference to a follow on talk the next day which would focus on integration tests for Seam2 using Arquillian, but I planned on being in the bpm strand all day and wasn't going to get to that unfortunately. It was quite a short presentation, stated reason being the dinner was near nevertheless it still packed a demo into the sesion.

The main thing that interested me was that the junit test case is actually uploaded and ran within the deployment container - I had to ask at the end what the full implications were for this (perhaps everyone else just implicitly understood) but the cool upshot is that at the point the test is ran you can get to anything that you would normally expect to see at runtime and check it's state. So controller throughput, cache states, etc.

This was coupled with jclouds which provides a vendor agnostic way of controlling machine instances. It does this by providing stub calls for things like startup, shutdown. Together you can use these to give you a simplified way of pushing your classes to an environment at runtime. The guys got a multithreaded test deployed to 5 EC2 instances and passing (demo glitch permitted!) very quickly with test code that focused purely on logic checks - no baggage in the code at all.

DAY 2
----------------------------
This was the day upon which I had planned the trip. If this reconditioned factory by a river in Berlin were to be my Olympus then the Workflow and BPM Track are to be my ambrosia.

The alternate track was covering jboss 6 and 7 theme talks.

Session 8
Kris Verlaenen & Koen Aers
jBPM5: Are your business processes ready for the future?
----------------------------
1, 2, 3, 5

This talk served as an overview of the upcoming jbpm 5 release, detailing some of the things that will be on offer in the new version and it's key characteristics. On display was the eclipse tooling for workflow planning, which showed BPMN2 support in it's graphical view. File generation was shown with the consumption of rules in the form of xml and another. Kris was the main presenter, who previously was a Core Developer on JBoss Drools before coming in to lead the development for this next release of jbpm.

From the top came on overview of bpm. In this context, a business process describes the order in which a series of steps need to be executed using a flowchart. It is at a high level or abstracted from the solution. In it's favour is agility and speed of development.

Jbpm5 is open source business process management. It contains a generic business process engine and supports native BPMN2. It purports to target business users and developers with a focus on collaboration.

Workflow is the core engine that manages your processes. It is minimal, generic, extensible and embeddable. The core engine is conceptualised into 2 parts; the runtime and definition.

BPMN2.0 is an OMG standard that defines

graphical notation
process definition format (XSD)
execution semantics

A demonstration was given showing an XML process definition being loaded into a runtime and transformed into a workflow in the graphical tool. We also saw a java representation which was good - showing that there are multiple ways to represent the same process. When the demo was ran the workflow visibly progressed in the eclipse graphical view with a marker denoting the current position.

Next up was a progression through a representation of the core engine that processes rules. The following picture was built up with an explanation of the components that add up to make the workflow in Jbpm.

Plenty of things to investigate here. Some things seemed more embryonic than others, for instance the best Google link I could see after a quick skim for WS-HT was here http://incubator.apache.org/hise/. I interpret this as stating that jbpm itself doesn't 'do' anything, the point is that these are the sorts of services you can use to leverage the core engine component model in your workflow - and ultimately make it do what you want in the way you want.

I'm hoping to hook up to the slides (http://www.jboss.org/events/JUDCon/JUDCon2010Berlin.html - post them guys!) so I can add some more context to the above image. I wouldn't want to delve too much into the deeper meaning with my knowledge as it is at this point. But I think there's still some value in there in that form.

A final point was made where the question was asked 'are your processes were ready for the future?' Traditional systems have problems with change, complexity and flexibility. Jbpm will give you adaptive case management, unstructured / non linear bpm and event driven bpm.

At wrap up were links to the #jbpm in codehaus, github/krisv/jbpm and installers/runtimes for jbpm5 that are in hudson - an ant script can automatically do most of what you need as in the version 4 release. Drools guvner is the knowledge repository in use. Here's a blog link from Kris detailing the release http://blog.athico.com/2010/10/here-we-go-first-jbpm5-snapshots.html

This was a fairly formal write up so what did I take from this? I suppose I got a bunch of terms that were new to me, pieces of a puzzle I can take away and muse over. As and when I come across these terms and ideas again I now have a visualisation of how these things slot together, where they go, why and what they can give.

Kris talks well about the topics in an engaging and enthusiasatic manor. I grabbed him for a chat before lunch to talk about the differences between Drools (and it's selection of sub projects), Jbpm Jboss Rule and Red Hat BRMS.
Kris's stance, as I understood it, is that Drools has components that feed directly into jbpm. Red Hat BRMS - I didn't get a straight answer on, other than it is a rules engine. Whilst Drools has a workflow component (flow) and a rules engine (guvner) these serve as feeder projects possibly into jbpm, or not if it doesn't benefit to do so. Think of them as playgrounds that can bear their own mature solutions or have things move into jbpm to improve it rather than competing projects that pull Red Hat's BRMS in a different direction. This seems a sensible stance to me.
Of note was the confirmation what Red Hat will not be productising version 4 of jbpm and instead providing migration support and development effort exclusively for 5. But, this also acts to douse water on the questions surrounding the imminent death of jbpm which came from some quaters of the audience (and proportedly the community at large).
I got the feeling that jbpm 5 will be a pull-together of pre-existing, tried and tested components for a more mature release cycle this time round.

Session 9
Eric Schabel
Get your BPM ducks in a row - preparing for migration to jBPM5
----------------------------
Schiphol won't eat your children

Eric's talk was good, concise and with some clear take away points.

He started off talking through where we're at with jbpm 3/4/5.

3.x was the first productised version of jbpm sold by Red Hat. It is tried and tested, proven to be solid in live environments. It has won awards (but I didn't write down the names so unless Eric's slides appear online I could not say which ones.). It will be supported until 2015.
4 was the start of a new product version. Missed getting into product at Red Hat, and now will never be supported commercially. Community only.
5 is the future, will be productised and supported commercially - hence a migration process from 3.x to 5 will be needed.

Eric showed and talked about examples of usage in the financial sector and government - no actual implementation details though (NDA's I guess). An interesting jbpm application he also talked about and showed a few slides for was the automated check-in pods at the Schiphol airport in Amsterdam. It was a good example of how real world factors could impact this system in it's deployed usage. When a user begins the process they place their bags in the pod which closes it's door to weigh your bags before printing your ticket and sending your baggage to handlers. Once a child climbed into someone's bags unnoticed, fortunately as the system uses a global transaction the check-in could easily be cancelled without any details making it into the system and they were returned before being put on a plane!

One thing that makes a jbpm 3.x->5 transition undesirably rough is that JPDL (http://docs.jboss.org/jbpm/v3/userguide/jpdl.html - the v3.x definition language) is not equal to BPMN2. Along with the following, it has been a focus area for people involved in preparing for the transition.

Your process initialisation layer - how do you start your processes? examples include web services, ejbs, or api calls - and when requested to start how do you prioritise their start (if they're going to be queued for example)? To ease migration implementing a one-stop shop first of all is the best way to go - for instance making all your processes start by web service calls (or whatever) means that calling systems only have one new invocation to worry about and your devs only have one mechanism to support.
A process definition file migration tool - not done yet but something which you could right click on for instance and turn into bpmn2 definitions would be desirable.
Something about 'bean script' from jpdl - guess this is a sort of embeddable run time thing that you can invoke, would not perhaps have a direct bpmn2 equivalent.
Process Implementations - do you keep to one single piece of code per process step or do you have reams of code in each node? Keep it simple and your transition will be easier - ie all your DAO goes in one code base which you talk to behind an interface. IMO this is pretty straightforward coding practice but I have never seen a JBPM install and implementation so perhaps it's not easy to follow this paradigm in the heat of the moment. Regardless, centralised process interaction is the key message here - if all in one place then changes are minimal -> apply OO principals to your process design practices:

methods==subflows
encapsulate==subflows
re-use (is POM up your process code into re-usable jar archives and allow people access via maven)
unit-test per node and sub flow
keep it all visible - despite what you may think, this is trying to say that you should talk to your customer in their language; show them things that they're used to seeing in your process definitions and use your sub flows to implement the technical detail - hide that stuff, keep the detail under water
exception handling - use a framework to handle your process errors - you don't want a single, stand alone process node on your workflow diagram acting as a 'go-to' statement for errors. 'fit the flow'

Migration tool needs to be able to do jpdl 3.2.x to BPMN2 CLI and api mapping. But is a community tool at the moment - however Eric strongly advocates customers / community members ask Red Hat for features or support if they need it. I take this as a pledge to create tools that have a target audience, not tools for the sake of it.

Roadmap is at http://community.jboss.org/wiki/jBPM5Roadmap and Eric will post slides at a later date.

Session 10
Tihomir Surdilovic
Applying JBoss Drools in Assistive Technology
----------------------------
Using rules to make cool tools

I couldn't take many notes in this talk, unfortunately Tihomir was stuck in Poland(maybe?), as such it was dialled in over Skype and so the pacing was not really conducive to writing and listening.

Some really interesting applications of rules based systems (drools in this case) in his research with people who have accessibility problems.

Assistive technology (AT) is targeted at 2 types of disabilities

Those with sensor disability - where they cannot process input from the world
And those with expression disability - they cannot provide feedback (output) to the world.

Slides to show that AT is a growing market that is helping people to live in and interact with 'our' world. However the user view is that lots of existing products and technologies are expensive, slow to market and out of touch with the needs of the users. Distinctions in AT compared to other markets are..

the needs are highly personalised and must be customizable (users very much know what's best for them in this case - and the range of variety in the requirements of the user base is extreme)
must be scalable (didn't really follow - maybe he means that once you've made one, you really should be able to replicate this success)
the users are the knowledge experts, not a focus group or otherwise.

Drools Expert, Flow, Fusion and Guvner gave a good business logic integration platform to start building raw and complex event processing systems/workflows for AT. Some of the experimental platforms like Drools Chance deal with imperfect information natively - which can help with variance in people's use of AT (for instance those with respiratory issues may not be able to use a system in regular a manor 100% of the time).

Drools' rule engine's live-rule deployment (where new rules can be loaded in at run time) helps users to formulate parameters and logic individual to their needs.

Things like the temporal event context in Fusion (and even the fact that events have context - ie a click event only makes sense when you're hovering over a clickable object) aids this development dramatically.

Session 11
Julien Viet - Advanced JCR Persistence in the GateIn Portal Framework
----------------------------
Interfaces repositories and my baseball cap

Sorry Julien, but this wasn't my thing. A good bunch of people in the room claimed to use GateIn / Portal, but I couldn't tell how it went down with them.

GateIn appeared to be similar to MS's sharepoint - pre built components that can be dropped into applications which the user can easily define. This is combined with things like content aggregation based on specs/standards like OpenSocial. Bridges to frameworks like struts/tapestry/portlets in Spring/Grails exist.

This all seemed interesting as it gives user defined content - always good, made me want to look into porlets more...

And then, after what seemed like only 2 minutes the talk banked left at the traffic lights into the world of the Java Content Repository (JCR) - but I missed why this is so pertinent. The abstract (http://www.jboss.org/events/JUDCon/JUDCon2010Berlin/agenda.html_ suggests that JCR is the persistence mechanism for your GateIn applications, but that it is difficult to deal with programatically (ie at runtime when your code wants to do things with it) as it is leaf/node based rather than giving you a straight up api to give you Strongly Typed Objects to play with. So Julien and his company have some tools that you can use to manage this - CRASH and Chromatic, which we saw a long demonstration of.

Session 12
Markus Doehring
Using jBoss Drools to Implement Guided Process Flexibility - Towards Efficiently Combining Workflow, Rule and Eventing Paradigms
----------------------------
These rules are made for breaking

There was a lot of content on these slides, and initially they were moved through quickly. I always end up wondering when I see this happen if the slides in question have been recycled from some other purpose.... Or equally that there's just a lot of ground to cover. Either way, I've taken more from the less encyclopaedic presentations the last few days.

Fundamentally Markus had some fairly straight forward things to say. His world is that of research and phD writing based on partnerships with local industry. He works for a company that analyse global market trends in IT to determine 'good ideas'.

The scenario he paints describes workflow rules as 'pull' paradigms and events as 'push' paradigms - he then asks when do we need each of these and what are the integration points with your engine.

He used his work on modelling ship maintenance systems as a Use Case starting point. A few things he raised were interesting.
He suggests a library of extensions to BPMN2.0 (but aren't these just user defined process nodes?) - things like timers that allow a flow to execution to continue unless a time bound constraint is breached. These are then re-usable through your processes.

Markus also advises that you perform data mining on your rules to determine when and how they're executing. Rules given by a business user are not always the truth (good!) - you need to be able to evaluate your rules, by analysing the stats you can learn decision trees that may be more efficient. "Unveil the power of the audit trail...". A wish list item for drools for him would be to have some sort of button to show you how and when your rules are matching your data. I hope to be able to revisit these ideas when I know more about bpm as a whole - it may be that these sorts of things already exist and they're just not used widely in practice.

An interesting point that someone raised in the discussion at the end was that the components being proposed are introducing uncertainty into the workflow - and that this is against the fundamental point of simple flows that can describe in atomic actions what will be happening as events are processed.

Session 10
Matt Brasier
Practical Enterprise Java Performance Tuning
----------------------------
Best practices now available to take away

Matt came along at the end of the second day to show off what a performance consultant's toolbox looks like. He was armed with some broken applications and wanted to show the diagnostic paths he takes when approaching unknown errors and performance glitches in systems. The sample applications were made available at c2b2.co.uk/judcon/sampleapp.zip (you may need to google around their site a bit - I haven't pulled down the applications myself).

This was an informative yet level and well received presentation, pretty much spot on for the last slot in the conference.

Matt started by demo'ing some of the features of the admin console and Jmx console. The slimmed down RHQ/jopr admin console (http://your-jboss-url/admin-console & login with admin/admin by default) contains a metrics tab in the memory subsystem that displays a breakdown of the jvm running your jboss app. This seems like a good starting point for checking on your system, but isn't going to set the world on fire just yet.

The jmx-console (http://your-jboss-url/jmx-console) is another built in component that is fast to use if you know what you're looking for and powerful if you're an advanced user - you could have your own mbean deployed which uses the jboss event mechanisms to generate heartbeat metrics and push them to a remote system for instance. Under the jboss.system.ServerInfo mbean is thelistMemoryPools operation - which gives you an ascii-art view of the memory usage of your jboss instance.

Both of these are interesting, but the better tools to use come bundled with the JDK install.

JVisualVM arrived with java 6 and is the successor to java 5's jconsole. It's pretty important that you install all the available plugins as these will improve it's out of the box diagnostic abilities.

Above: the visual GC plugin showing the breakdown of a groovy script which is giving the string pool a kicking

There are also some interesting command line tools in your jdk bin directory

jps will show pids for java
jstat will show garbage collection stats
jstack will show thread dumps, which is good for seeing what your threads are up to at certain moments in time - what methods are being called (if lots of the same method constantly appear then it's possible that is a slow method - gives a starting point to look at) and locks held by these threads
jhat dumps the heap

After showing these diagnostics tools, next up was some testing tools; if you cannot reproduce the problems that are occurring in a controlled fashion then you're going to have fun trying to determine their root cause.

Jmeter - apache load balancing tool that can be scripted to replay web requests to your applications. Can record your actions, can run in a distributed fashion (good for placing in cloud provisioned systems around the globe and playing your requests
The Grinder - another load testing tool based on Jython. This allows you to write jython testing scripts (v. powerful but can get complicated), it can still record these scripts from your actions which gives a good starting point. Better for more extreme testing scenarios.

Followed were some demos of Matt's practices in action.

With the first troublesome application deployed to jboss he began by looking in the Server.Info mbean in the jmx-console and looked at the thread dumps. Theory being that if a thread was up to anything interesting it's dump would be quite long, whereas other threads would be sat in the pool. These showed the method calls in action which gave us an offending piece of code in the application's startup servlet.

Unfortunately I forgot to note down what the remaining 2 issues were that Matt resolved, but I did take details on the processes he followed.

In one case he used visualvm to monitor the thread statuses (sleeping, waiting, etc) and memory/api calls being made. Coupled with visualgc this gave one view as to what the java process was doing.

In another just the command line tools were used (jstat can output per second <or your unit of time>!) to monitor the cause of garbage collections - we were able to see a large number of garbage collections being invoked by the user, each taking a few seconds (the reasons for this could take up another future blog post if people are interested) and thus slowing down our application. One way to fix this without code modifications would be to intialise the application with the -X flag that disables user GCs.

An interesting Q&A followed, one choice piece of advice was to not fight the jvm too much unless you knew what you were doing - as it's pretty well tuned for most use cases out of the box.

Final Thoughts.

So there you have my take on the presentations I attended. Some I took technical detail from. Others were more of an introduction to a field I had not yet come across. All in all it was a good success in my books, with some cutting edge tech on display by leaders in the field.

The venue was pretty neat it must be said, 2 days was a great length too - I've attended longer / shorter conferences in the past. A single day can seem tight, too many days and travellers fatigue can kick in. Free beer too - thanks Ingress :)

The presenters I spoke to were all accessible and open for discussion - which was really good for the inquiring amongst us.

What would I like to see improved? I think that in some aspects, a hackfest doesn't live up to it's name if it is too organised, but a little more planning here would have helped integrate the attendees more.

Overall however I should like to congratulate the organisers for hosting a successful event. On to next year :)

My Blog

Wednesday 5 January 2011

Centos netinstall location information

Saturday 1 January 2011

Add JDK javadoc in netbeans 6.9.1

Tuesday 30 November 2010

Who's driving this thing?

Tuesday 19 October 2010

JUDCon 2010 - Details & Thoughts

Tuesday 12 October 2010

Hello World