J and I and Me
JAX Preview
This year's
JAX will see some interesting sessions that I would like to highlight:
- The New School Enterprise IT Day will show how new technologies and business challenges will change the Enterprise IT. I am quite happy that this has been added to the JAX schedule - because I believe there will a huge shift in this area in the next few years.
- The Cloud Computing Day will show the latest and greatest in the Cloud. Several topics - such as the different Java PaaS alternatives but also IaaS will be explained in detail.
- I have done several Code Retreats at adesso AG and always found them to be a great experience for all involved. Therefore I am pleases that I can do a Code Retreat at JAX this year.
- And of course the Advanced Spring Powerworkshop will take place - it is a unique opportunity to dive deeper into the framework.
Some of my colleagues are also presenting. For example Alexander Frommelt will talk about
IT landscapes and also about
Portals and whether Portlet are really a good fit for them. Halil-Cem Gürsoy will talk about
Google App Engine.
So I am really looking forward to the event - and would be glad to meet you there!
Common Misconceptions: The Waterfall Model
I think the Waterfall Model is the result of a big misunderstanding, probably one of the worst in out industry.
Look at Royce's original paper (PDF can be found
here. You will notice that the paper starts with the separation of different activities such as analysis and coding. To me that sound like an attempt to actually define basic software engineering activities instead of just unstructured hacking. The paper goes on and discusses more different phases that a project might go through. It shows a figure pretty much like the Waterfall model we are used to. No surprises so far.
But then the fun starts: The third figure already shows that the steps are not necessarily performed in order. The text says:
... as each step progresses and the design is further detailed, there is an iteration with the preceding and succeeding steps but rarely with the more remote steps in the sequence.
Let me repeat: The original Waterfall paper says that you might need to go back to previous steps, even remote ones. It even uses the term "iteration".
It goes on and discusses that once you run in production you might learn that your system does not perform well enough. That leads to major problems - and you will probably go back to the analysis. You might call it an iterative approach - even though it is probably not voluntarily.
Even better: The paper suggests:
If the computer program in question is being developed for the first time, arrange matters so that the version finally delivered to the customer for operational deployment is actually the second version in so far as critical design / operations areas are concerned.
So essentially you should do at least two iterations - the first version will not get it right. Another hint at an iterative process.
And the paper even suggests to involve the customer - probably one of the most important points in Agile practices.
Of course the paper includes sections that are quite different from the Agile school - such as the focus on documentation. But it is from 1970 - and the author is specialized in systems for spacecrafts. Those still rely a lot on documentation even today because of the extreme risks those systems have.
However, the bottom line is that the original Waterfall paper does not advocate what is now considered the Waterfall Model. It does not require going through the steps in order and it even mentions that the first release will not be a good solutions. Quite contrarily: It talks about iterations - very limited of course but it hints the direction Agile and iterative processes took later on.
I am still confused how the industry was able to misunderstand this paper. I wonder how much damage it did. Even today people still talk about Waterfall. I think everyone working with software processes should read this paper. I also suggest to use the term "Misunderstood Waterfall Model" when discussing a model that suggest going through the steps in a strict order. Because that model is just a misunderstanding, it is not what Royce described.
Oh, and next time someone talks about the Waterfall Model - don't forget to ask him or her whether he has read the original paper about it...
Some Possible Enhancements for Java and Java EE
The recent discusssion about Java EE and Spring made me think. Actually I find it odd to fight over the programming model used to develop Java applications. Neither Java EE nor Spring will go away - the market shares of both are just too large. At the end everyone involved in the discussion is trying to sell a platform i.e. an application or web server. That product must offer good support for both programming models - and all the others like Scala, Groovy etc.
IMHO the programming models have a lot in common now. Spring also covers advanced challenges like Social Media, NoSQL etc - see my older blog post
Spring vs. Java EE and Why I Don't Care. So it is not in the field of programming models where innovation is laking. Instead we should focus on improving the platform.
So what is it that would really make Java a better platform? Here are some ideas:
-
Ever since the invention of Java EE roles like the Application Assember were defined. I never saw a project that followed this workflow. Nowadays there is just an automated process that results in an EAR or even a complete setup including an application server. It is time to face this reality and get rid of these role descriptions. The deployment process is not done that way. Java needs to step away from it.
-
Nowadays application server are seldom used to host more than one application. So maybe that offers some possibilities to improve the deployment processes and also the features. If there is just one application on a server anyway the strict isolation between applications Java EE offers does not make that much sense any more.
-
It would be good to focus more on deployment during the development process. Applications are much more often deployed into testing or development environments than into production. I am wondering how many CPU and I/O cycles are wasted compressing WARs and JARs and then EARs during a build process - and immediately decompressing them on the web or application server so testing can start. Can't the Java EE standard do something about this? Obviously there are exploded WARs and EARs - but they are not used as often as they should be. Obviously there are tools like JRebel and Play - but what if the complete tool chain including IDEs, build tools, servers etc would support a standard here? What if all of those parties could collaborate to make this work flawlessly? I think we would see quite a boost in developer productivity.
-
Often I see applications modularized in several WARs. These modules need to talk to each other. However, there is no easy way to do this. Usually they end up using Web Services or another HTTP based protocol. Compared to a local method call this adds the overhead of HTTP, sockets and marshalling / demarshalling. A possible solution would be OSGi - but that technology is quite complex. The reason to me seems to be the problems around shared class loading. Classes that might represent parameters or return values can be shared between modules. JBoss includes a solution based on JMX that did not support shared classes - so modules need to have the correct classes available. This is the same as for Web Services or the other HTTP protocols. But there is no standard apart from JMX - which is very low level. Can't we have some communication between WARs that in the standard? Maybe based on JMX? Or at least best practices around it?
-
Concurrency in languages like Erlang is very different from what we are used to in Java. There are actors that work on a serial list of messages. Concurrency is achieved by multiple actors working in parallel - while the actors themselves are just working serially. This approach is also used by the Scala library Akka on the JVM. In Erlang this approach leads to lots of actors - far more than the number of threads that can be supported on the JVM. If the JVM wants to stay relevant for this model we need proper support for it.
-
Actors in Erlang and in Google Dart - they are called Isolates have separated heaps. Those heaps can be independently GC'd - so while one actors spends time doing GC the other can continue to work on messages. By design you end up with less stop-the-world-GC. Maybe we could have a similar concept in Java? This is an extension of the previous suggestion but there might be other options as well. For example each WAR file could have a separate heap.
Those are just some - possibly crazy - ideas. My intention is to focus on making the Java platform better for all of us. And I believe for that the discussion around the future of Java must focus more on the JVM and the infrastructure like web or application server than on programming models. There is quite some innovation in the area of the programming models anyway. For example there are lots and lots of Open Source projects - frameworks and complete languages. But for infrastructure standards are needed - so any server and environment can work with the solution. So that is what the standardization process should focus on.
What do you think? I am looking forward to your comments!
Testing: Again
As you have probably noticed, the last blog posting
Testing Considered Harmful was actually a April Fool's Joke. I believe Test Driven Development is one of the most important innovations in the last years. I have been infected ever since Kent Beck explained JUnit to me.
Here are some things adesso does to make TDD a reality:
The list could go on. So please program - and drive - safely! If you were caught by the April Fool's Joke: I hope you don't mind. If you do: I apologize.
Testing Considered Harmful
Unit testing started with JUnit - and was infectious. As you can see in
JUnit Test Infected: Programmers Love Writing Tests that was the intention from the very beginning. And so the infection spread.
Nowadays almost all programmers do extensive Unit Tests. And therefore we see problems arising. Instead of thinking about a problem and building a reliable solution developers now just write a test and code away. At one point they get a green bar and consider the job done. But can you build proper software that way? Dijkstra stated in 1972: "If you want more effective programmers, you will discover that they should not waste their time debugging, they should not introduce the bugs to start with." - see
Wikiquote. I think the same holds true for testing. So instead of the ever famous code-test-refactor cycle we could use more of the
Feynman Way of Solving Problems. If you really think about the solution first you will find you won't need a test. Instead you will actually write correct code from the beginning. You won't need to rely on tests. They cannot show the absence of bugs anyway - as Dijkstra mentioned. In the end you will end up with cleaner code and less code - as there are no tests to be written.
This principle can be applied to other areas as well. I think one reason why there are so many car accidents are the very effective brakes cars have now. If we had less effective brakes people would think about possible problems beforehand, drive more carefully - and we would have less accidents.
Dangerous Code Example in EJB 3.1 Spec
I am currently working on a training concerning Java EE Best
Practices. For that reason I am reading through quite some material
about Java EE and EJB. However, looking at the EJB 3.1 spec I realized
that the code examples are an example of rather bad coding practice.
As an example here is the original code from EJB 3.1 Spec p. 348. It is supposed to
show how
UserTransactions can be used:
@Stateless
@TransactionManagement(BEAN)
public class MySessionBean implements MySession {
@Resource
javax.transaction.UserTransaction ut;
@Resource
javax.sql.DataSource database1;
@Resource
javax.sql.DataSource database2;
public void someMethod(...) {
java.sql.Connection con1;
java.sql.Connection con2;
java.sql.Statement stmt1;
java.sql.Statement stmt2;
// obtain con1 object and set it up for transactions
con1 = database1.getConnection();
stmt1 = con1.createStatement();
// obtain con2 object and set it up for transactions
con2 = database2.getConnection();
stmt2 = con2.createStatement();
//
// Now do a transaction that involves con1 and con2.
//
// start the transaction
ut.begin();
// Do some updates to both con1 and con2. The container
// automatically enlists con1 and con2 with the transaction. stmt1.executeQuery(...);
stmt1.executeUpdate(...);
stmt2.executeQuery(...);
stmt2.executeUpdate(...);
stmt1.executeUpdate(...);
stmt2.executeUpdate(...);
// commit the transaction
ut.commit();
// release connections
stmt1.close();
stmt2.close();
con1.close();
con2.close();
}
... }
What is wrong with this code? Well, first of all it does not
compile. Quite honestly I wouldn't really care too much about that - an
example is fine as long as the important point is still brought
across. But in this case it does matter as we will see later on. So here is
what the method should read like:
public void someMethod() throws SQLException, NotSupportedException,
SystemException, SecurityException, IllegalStateException,
RollbackException, HeuristicMixedException,
HeuristicRollbackException {
In the original example any code concerning Exceptions
has been left out. So what happens if an
Exceptions is thrown? No
close() is ever called and therefore none of the resources are ever cleaned up. Let's fix this:
@Stateless
@TransactionManagement(TransactionManagementType.BEAN)
public class MySessionBean {
@Resource
javax.transaction.UserTransaction ut;
@Resource
javax.sql.DataSource database1;
@Resource
javax.sql.DataSource database2;
public void someMethod() throws SQLException, NotSupportedException,
SystemException, SecurityException, IllegalStateException,
RollbackException, HeuristicMixedException,
HeuristicRollbackException {
java.sql.Connection con1 = null;
java.sql.Connection con2 = null;
java.sql.Statement stmt1 = null;
java.sql.Statement stmt2 = null;
try {
con1 = database1.getConnection();
stmt1 = con1.createStatement();
con2 = database2.getConnection();
stmt2 = con2.createStatement();
ut.begin();
stmt1.executeUpdate("");
stmt2.executeQuery("");
stmt2.executeUpdate("");
stmt1.executeUpdate("");
stmt2.executeUpdate("");
ut.commit();
} finally {
if (stmt2 != null)
try {
stmt2.close();
} catch (SQLException ex) {
}
if (con2 != null)
try {
con2.close();
} catch (SQLException ex) {
}
if (stmt1 != null)
try {
stmt1.close();
} catch (SQLException ex) {
}
if (con1 != null)
try {
con1.close();
} catch (SQLException ex) {
}
}
}
}
This is the infamous try-catch-finally-try-catch block. The
null
checks actually avoid calling
close() on objects that have not been
created - as the creation might already have failed and and exception might have been thrown. As the
close() operation might throw an exception this also needs to be handled. IMHO it is OK to swallow the exception - there is not really anything that can be done as the resources are already being closed. Using Spring's
JdbcTemplate would have avoided the problem as the resource handling
is done by the template then. I would strongly recommend it in code like this. It can be used independently from the other parts of Spring - e.g. Dependency Injection can still be done by Java EE. See
http://static.springsource.org/spring/docs/3.1.x/spring-framework-reference/html/jdbc.html#jdbc-core for details.
Now there is another thing that is still not OK - actually the most
important point. The
UserTransaction is never committed nor rolled back
if an exception occurs. Let's fix this, too:
@Stateless
@TransactionManagement(TransactionManagementType.BEAN)
public class MySessionBean {
@Resource
javax.transaction.UserTransaction ut;
@Resource
javax.sql.DataSource database1;
@Resource
javax.sql.DataSource database2;
public void someMethod() throws Exception {
java.sql.Connection con1 = null;
java.sql.Connection con2 = null;
java.sql.Statement stmt1 = null;
java.sql.Statement stmt2 = null;
try {
con1 = database1.getConnection();
stmt1 = con1.createStatement();
con2 = database2.getConnection();
stmt2 = con2.createStatement();
ut.begin();
stmt1.executeUpdate("");
stmt2.executeQuery("");
stmt2.executeUpdate("");
stmt1.executeUpdate("");
stmt2.executeUpdate("");
} catch (Exception ex) {
ut.setRollbackOnly();
throw ex;
} finally {
ut.commit();
if (stmt2 != null)
try {
stmt2.close();
} catch (SQLException ex) {
}
if (con2 != null)
try {
con2.close();
} catch (SQLException ex) {
}
if (stmt1 != null)
try {
stmt1.close();
} catch (SQLException ex) {
}
if (con1 != null)
try {
con1.close();
} catch (SQLException ex) {
}
}
}
}
So when a problem occurs the transaction will now be marked as
rollback only and ultimately rolled back. If there is no exception it will be
committed.
I hope I made no further mistakes in the code - let me know
otherwise. In Spring resource handling is done for you by the
Templates and therefore I might be doing something wrong here. Using
that approach would also have made the code a lot less complex.
So why this blog post? Bad resource handling and transaction
handling is far too common. I have done a lot of reviews and usually
Enterprise Java applications fail to do a good job in this
regard. This is dangerous, because
- The whole point of transactions
is to provide safety if something goes wrong. If you do not commit nor
roll back the transaction it will be left in an undefined state and all kinds of things might happen. This is
compromising the safety of transactions. So the transactions become
more or less
useless. You might as well just not use them at all.
- Not closing connections and statements will eventually lead to
resource leaks e.g. connection pools running out of connections. This
will at one point make the system fail.
- These problems are hard to detect - they will only surface if
exceptions are thrown. So finding and eliminating the problem is
complex. You should rather get it right from the start.
Sadly the EJB spec seem to do resource handling constantly wrong. The EJB spec is
not the only document that does resource and transaction handling
wrong. Quite the contrary: I have a read a lot of tutorial and other
documents that get this entirely wrong. However, IMHO at least the
example in the spec for using the
UserTransaction should show how to
use it in a bullet proof manner. This document is read by many
developers and other authors. Bad practices in such a document might
end up in a lot of code and other publications.
Random Thought about Google Dart
Here are some thoughts about Google Dart and its possible usages. They are just wild ideas, I have no information whether they will actually be implemented or whether they are shared by anyone at Google:
- Compiling Dart into JavaScript is a good idea. It means Dart is immediately usable on any platform. There is no other way to quickly get it out to so many developers. Users are not quickly to switch to new browser - think about IE6.
- I originally expected that a Plug In and VM for Dart on Chrome would be released quickly. Now I think that would have been a bad idea. Chrome would have had a good support for Dart while the rest of the browsers would have been left in the cold. They would probably not use the Google VM - remember that they also have their own JavaScript VMs. It is also unlikely that they would create their own Dart VM. So the current approach create a level playing field and makes it more likely for Dart to succeed.
- I think the Dart VM might be an option for Android - maybe not in the near term but probably in the long run. Also it might be enough for Google to have an alternative VM. It just means it has a stronger position if a deal with Oracle must be closed.
- Dart is a good fit for Google App Engine (GAE). Isolates have just one thread - that is also what Google App Engine enforces. That has the advantage that GAE can restart isolates. Isolates can be snapshotted and migrated to other VMs - which is great if you want to react to different load by offloading work from a machine. Isolates can only communicate via ports - that allows you to minimize dependencies to the outside. Again that is great if you want to restart or migrate Isolates. It might also be a way around the limitation that GAE won't allow you to open sockets. Java without Multithreading and without the possibility to open sockets is hard and no fun - and that is why I believe Java on GAE ist not that great. This changes dramatically if you use Dart.
So we will see where this is headed. Remember the first days of Java: It was seemingly all about Applets. Now it is all about the server. Technologies might succeed in other areas than originally planned.
If you are new to Dart - here is a presentation about Dart:
Labels: Dart, Google Dart