Six steps to JBoss (and Tomcat) with IIS 7

I've had a devil of a time getting IIS 7 talking to JBoss. It took 3 different sites and flash of insight to get all the settings working. I'm not a system administrator, and I'm not an IIS expert, so I may have missed a default somewhere or gone further than I needed to. Here's what I did, in hopes it helps you.

Java Components and Package Visibility

I have a recurring problem: I want to build a component within a large application. Here's an example: I want to synchronize QuickBooks Invoices with my application's invoices. The component API produces and consumes application invoices, sales reps, accounts, and customers, but the internals of the sync task are very complicated, involving 3rd-party Invoices, maps to interface invoice-like objects, synchronizers, etc. The main app has no need to know most of this stuff.

Java provides no mechanism to build a subclass structure for my sync component--I must either build the whole thing in a single package containing hundreds of package-scoped objects, or I must build public objects visible to the rest of my app so my component can have an internal package structure which models its behavior.

What I really need is a visibility modifier making classes only visible to their sub-packages.

Creating Healthy Craftsmen

There's been a healthy debate over on the software craftsmanship mailing list about what craftsmanship means in the context of software development. I've been a proponent of, among other things, some set of standards by which a practitioner's capabilities can be measured. That's been unpopular, as you might imagine.

Documentation in the wild

I just started a new project.  I has no documentation except the code.

In this particular project, this is generally a Good Thing--the code in question is clear, well written, and (ahem) completely without comments.  I'm not having a lot of trouble understanding it, overall, because it really is very good code.  But... I've spent 2 days trying to figure out how one important feature works.

http://thedailywtf.com/Articles/Documentation-Done-Right.aspx

'Nuff said.

Kanban in IT

Here's a nice series of articles on using Kanban in an IT setting. I found it interesting because it shows not only what they did, but how their process evolved over time in several settings. http://blogs.lessthandot.com/index.php/ITProfessionals/ITProcesses/applying-kanban-to-it-processes-part-1

Sucking Less: Checking In More Often

I'm fairly fearless when coding, which means that about once a week, I delete a huge chunk of something I should've kept, or change something into something unrecognizable, thereby inadvertently breaking a dozen unit tests. When I discover the problem, usually about 4 hours later, I no longer have any idea what I did that made the bad thing happen. Then I spend another 2 or 3 hours figuring out what I broke and fixing it.  Ugly.

On my personal projects, I check in code every time I get a unit test working. My checkins are something like 15-20 minutes apart.  On projects I get paid for, though, checking in means running the whole unit test suite, and that can take 10 minutes (on a good project) or 2 hours (on a bad one)--so I don't do it very often. That's when I get into trouble.  I've been meaning to solve that problem for some time, and Joel Spolsky's blog topic last Wednesday (Joel on Software) finally kicked me in the pants.  It took 15 minutes to solve the problem; here's how I did it.

Annotating Custom Types in Hibernate

Hibernate has a lot of nice features, and it's pretty well documented, but a recent need to add a simple custom type to an existing mapping left me flailing around for documentation on exactly how to do it. I wanted to do it with annotations, not by updating the Hibernate configuration (that approach is well-documented). Here's how it's done.

java.util.DuctTape

Overview

The proposed class, java.util.DuctTape, is designed as a general purpose fix for a variety of commonly-observed situations in production code. It serves as a temporary patch until a permanent solution is developed and deployed.

Database/Code impedance mismatch

I love natural keys in database design. You have to pay attention, though: the natural impedance mismatch between a programming language representation and the database representation of the key can bite you.

Consider an object whose primary key might contain a date--say, a change log record. Oracle and DB2 both store a DATE as a time containing year, month, day, hours, minutes, and seconds. No timezone. The natural mapping for a Java tool like Hibernate is to map to a java.util.Date, which stores the Date as a time in milliseconds since the epoch GMT, and then maps it to whatever timezone is set on the machine where the code is running for display and conversion.

Now consider what might happen (especially if our change log record is attached to some parent object);

1. We create and save the object; it is persisted. The local cached copy contains a non-zero value for milliseconds, but the database has truncated the milliseconds value and saved it.
2. Later on in the code somewhere, we have reason to save the object again, perhaps as part of some collection operation.
3. Hibernate looks in its cache, compares it with the database, and notes that the values of the Date don't match--so it tries to save the value again.
4. The database dutifully tosses out the spare milliseconds, and bam! we have an attempt to re-insert an existing record, so it throws an exception.

This is all terribly confusing to the programmer, who, inspecting the objects in question, sees no difference between what's in the database and what's in her code, especially since the default display characteristics of her database browser and her debugger don't show the milliseconds.

The easy fix in this case is to declare a class which matches the database representation--in this case, a good choice would be to declare a new class which truncates the milliseconds. A modest example is shown below:

/**
* Public Domain; use or extend at will.
*/
import java.util.Date;

public class DbDate extends Date {
/** increment if you change the state model     */
private static final long serialVersionUID = 1L;

/** @see java.util.Date#Date() */
public DbDate() {
    long t = getTime();
    setTime(t - t%1000);
}

/** @see java.util.Date#Date(long) */
public DbDate(long t) {
    super(t - t%1000);
}

/** @see java.util.Date#setTime(long)     */
@Override
public void setTime(long time) {
    super.setTime(time - time%1000);
}
}

Also note that if you declared the database column as a TIMESTAMP, the Java and database representations more-or-less match--avoiding, in this case, this kind of problem. Note that Oracle doesn't support TIMESTAMP_WITH_TIMEZONE in a primary key, and DB2 doesn't implement TIMESTAMP_WITH_TIMEZONE at all--as of the last time I had access to DB2.

Dealing with timezones is another topic entirely--one which I'll take up in a future post.

Limiting Irreversibility

This afternoon I was reading Martin Fowler's commentary on architecture: http://www.martinfowler.com/ieeeSoftware/whoNeedsArchitect.pdf, and ran across the following:

At a fascinating talk at the XP 2002 conference (http://martinfowler.com/articles/xp2002.html), Enrico Zaninotto, an economist, analyzed the underlying thinking behind agile ideas in manufacturing and software development. One aspect I found particularly interesting was his comment that irreversibility was one of the prime drivers of complexity. He saw agile methods, in manufacturing and software development, as a shift that seeks to contain complexity by reducing irreversibility—as opposed to tackling other complexitydrivers. I think that one of an architect’s most important tasks is to remove architecture by finding ways to eliminate irreversibility in software designs.

I think he's absolutely on the mark with this--in fact, I think it illuminates one of the two or three key roles of architecture in system implementation. If the architect focuses on helping to define approaches which are hard to change once the system "complexifies", and in helping developers write solid code that can easily be modified when needs change, (s)he goes a long, long way towards making the system flexible and maintainable.

Note that there's two architectural roles there: (1) helping to make key decisions early, and (2) mentoring developers around those decisions.

On most of the dev teams I've worked with, the first set of decisions is made by proposal and refinement--one of us proposes an overall approach, and the rest of us point out tweaks or whole new approaches until the whole thing gels in everyone's mind. It seems to work, if everyone is engaged. Some architects would prefer to "rule by fiat", but I've found that generally results in systems which can't be maintained or in far more work than is needed. It's very hard to get key decisions right by yourself. To illustrate: I once proposed a relatively modest refactoring in a large thorny user interface, to separate business logic from display logic and generally make it easier to maintain. The reigning architect decided we needed a complete rewrite, in direct opposition to the opinion of everyone else on the team. Nobody objected strongly, though everyone quietly agreed that a rewrite probably wasn't needed--it's lot more fun to write new code than to modify old. Nobody asked what the minimum effort needed to meet the requirements was. About two years and a couple million dollars later, the new system is, indeed, quite a bit better-structured than the old one. It's not clear if the new UI will produce allow faster, cleaner updates than the old one--but it sure cost a lot to build: about twice the initial estimate, and about 6 times the original proposal. (By the way: the rewrite is considered a success by all involved. See "What Could Possibly Be Worse Than Failure?" by Alex Papadimoulis.)

A second role implied in minimizing irreversibility is "mentor". Writing good code is hard; writing well-structured code without someone to bounce your design off of is doubly hard. I spend a lot of time talking with the members of my teams, trying to make sure we have a design that's flexible and understandable. A lot of what we think of as "architecture" starts out as a small feature being implemented by a relatively junior developer. I try to make sure (s)he has someone to work with in the early stages of that work, or at least a trial design to start from.

I love Martin Fowler's writing: he always gives me something good to think about.

SEMAT and development principles

My first reaction to SEMAT was--is this practical? But as I've thought about it more, I've decided there are some principles of good software design and implementation they can probably agree upon and illuminate.

For context: I spent 15 years as a practicing nuclear engineer before becoming a practicing software developer.

"Engineering practice", as I found it in the field, is often as arbitrary as "software development practice". What is "good" is measured first by "what works", second by "what's elegant", and finally by "what's inexpensive", and is judged primarily by senior practitioners working against their own experience rather than some set of objective standards (though those exist as well). In hardware engineering, the time lapse between design review and implementation is quite often very long (especially large-scale design, e.g. power plants, where I worked). As a result, feedback loops are even longer and harder to manage. What dominates in the large scale seems to be "what worked"--and just as often, "what failed".

This seems to me to be exactly the type of thing we're developing now as developers--we now have lots of categories of languages for solving different types of problems and we're developing solid tools and techniques for measuring performance, managing projects, and closing the loop between design and implementation. The world for software developers is a FAR less arbitrary place, and has far more development of tools and techniques, than in 1976 when I started coding.

The body of common practice is regularly changing in hardware engineering, as analysis tools get better. When I graduated, one of my first tasks required a stress analysis, which I did with a calculator by hand. These days, an engineer would set that up in a desktop finite-element analysis program with a nice UI, and he'd get a better answer in less time. The principles are the same, though: determine, through an understanding of material behavior and machine design, what a specific application required of the machine and what combination of off-the-shelf components and custom machining could be used to implement that application. It's very much what I do today: I pick off-the-shelf components and custom components to create a design to meet certain requirements.

Can we, as a group, specify some of the principles on which those decisions get made? I suspect we can. While I'm only really fluent in one sub-part of OO design, I know of a few; consider the SOLID principles in OO design, various common design patterns, and the corpus of tools and techniques in Knuth's "The Art of Computer Programming".

Coupling Design and Implementation

Six weeks ago or so, our development team reviewed a small design change in the way status is managed by an object. Basically, we broke one state variable into four, and thought more carefully about the state transitions allowed and expected for the class in question.

Yesterday, one of our analysts, at the prompting of one of our developers and two of our data designers, reviewed a completely new take on the same design change... none of the guys in question knew about the previous design change (they'd missed the design review). We ended up going with the previously-reviewed design.

The whole meeting and the thought that led up to it could have been avoided if there had been some mechanism for ensuring the approved design was implemented. This bit of design, like all design, is pretty much wasted if it remains in design documents.

One way to handle this would have been to somehow automatically compare the existing design artifacts to the implementation to see if they matched, and complain if they didn't. Even if we didn't implement the approved design right away, then, there'd be a mechanism for reminding developers that they planned to do something one way, and haven't made it happen yet.

Three snippets of "interesting" code

Here's a bit of code I use in interviews:

try {
  if (foo())
    return 1;
} catch (Exception e) {
  throw e;
} finally {
  return 3;
}

The questions I start with are:

1. what does this do if foo()==true? If foo()==false? If foo() throws an exception?
   


2. how could you recode this more simply?
   


3. the original spec was:
   
   • if foo is true, return 1,
   
   
   • if foo is false, return 3,
   
   
   • propagate exceptions.
   
   

Provide code to implement the spec.

It's sobering to note that over 2/3 of interviewees for senior Java programmer positions fail all three questions. How would you answer them?

I firmly believe that even mid-tier developers should have no problem describing what they mean in code, and in understanding what others mean, even in code which is poorly written. There always seems to be a snarl somewhere that nobody wants to touch (I've written one or two of those myself). For the most part, though, the bad code I see is the result of "I'm not sure how to do this, but this seems to work".

Here's one such snippet:

static final BigDecimal ONE_HUNDRED_PERCENT = new BigDecimal("1.00")
    .setScale(MathUtils.DOLLAR_SCALE, MathUtils.STANDARD_ROUNDING_MODE);

This is just bad code, unless you're letting BigDecimal manage all the results' scales itself (we aren't, and you shouldn't; see my previous article on BigDecimal rounding). It's completely replaceable with "BigDecimal.ONE". It leaves the resulting code less clear, and performs no useful function.

I found the following code (paraphrased) in a Java application I'm working on:

foo(vc.getNewValue(), ppCorrectedValue=vc.getNewValue());

It's perfectly correct and exactly equivalent to:

ppCorrectedValue = vc.getNewValue();
foo(vc.getNewValue(), ppCorrectedValue);

That sort of expression is common in C; most C developers are aware that the equals operator evaluates to the left-hand value of the expression. Many Java developers aren't aware of this fact--so I was surprised to see it. I'm not sure it improves the readability of the code, though, so I'm refactoring it into the second form above.

Where I think the equals operator behavior really helps in Java is when setting up initial values:

i = j = 0;

Good code is hard enough to write; have pity on the next guy, and be as straightforward and clear as possible. Arabesques like assignment inside a function call parameter list just make your code harder to read and maintain.

Introducing SEMAT

I'm trained as a nuclear and mechanical engineer. In those disciplines, there's an underlying body of knowledge and practice for design and implementation, based on a couple hundred years of experimentation and data analysis, on what works and what doesn't, and why.

Software "engineering" as it is practiced where I usually have worked, seems to be 75% gut feel and 20% anecdotal experience, with the remaining 5% subject to real analysis. I am committed to bringing at least some of the discipline of field engineering (those parts I understand well, and can adapt) to the practice of producing software.

A group of the leading lights in methodology and architecture have come together to form SEMAT -- an acronym of "Software Engineering Method and Theory", with the goal to "refound software engineering based on a solid theory, proven principles, and best practices". It would sound like so much smoke, but consider some of the signatories:

Scott Ambler
Barry Boehm
Bill Curtis
James Odell
Ivar Jacobson
Erich Gamma

The list is quite long, and luminous.

I suggest you go have a look: http://www.semat.org. I'm looking forward to finding out what Scott Ambler and Ivar Jacobson can agree on.

BigDecimal

Whenever you're working with money in Java, you should be using BigDecimal. There are other applications, but far and away the most common is monetary calculations. The biggest reason is accuracy: floating point values (stored using IEEE standard 754) can't represent common decimal amounts. For that, you need actual arbitrary precision math--the same kind you learned in grade school. That's what BigDecimal provides. Yes, the operations are clumsy-looking and require a lot of typing. Wouldn't you want your bank or your tax office to have full control over the precision of the values they use to calculate your interest or taxes? You owe it to your users to provide the same precision.

Design Constraints

Last night a bunch of us discussed the lines between architecture, design, and implementation.

I started with a working definition John Prentice gave me--heavily paraphrased, from memory: architecture is "enough design that it's clear the system requirements can be met", and design is "enough design that the path code will follow is clear". Bryan Helm suggested that architecture should also constrain design so that "good design choices will be the natural ones; bad design choices will tend to be excluded", theory being that architects tend to be senior designers as well.

There's a lot of good discussion of "what it is" on the web; I've provided some links at the end for articles I found particularly illuminating and useful.

Within the context of executable design, the group of us agreed we'd all been dealt horrible or irrelevant architectures and designs, and one hallmark of a good architecture, and also a good design, is that you could rough out code that followed the architecture, and it would execute as expected. Similarly, you could implement the design exactly, and it would execute as expected. In both cases, obviously, the system would be grossly incomplete--but the specification at the design level would be complete enough that all the functions required of the system were stubbed out and ready for detailed implementation.

I'm not sure the distinction between architecture and design is important from the point of view of a set of executable design tools. I think architecture should constrain design, and design should constrain construction. Given the way code is actually constructed, the architectural description and the design description must be maintained as the software is developed. What better way to do that than making key portions of these descriptions the armatures upon which code is developed? Code generation isn't the way to go, because you have to keep round-tripping, and you lose the abstraction provided in the design level.

The only way I can see to allow the design to constrain the construction is to use the design itself as the runtime environment for the final code. In the Java and .NET worlds, this means custom class loaders sitting between the developed code and the design, verifying conformance to the design and allowing the design to execute directly if code hasn't been provided to implement it. In this way, you can actually test the design as it is developed, in the same way you test code as it is developed: by running it.

There are many good articles; here's a jumping off point:

http://www.ibm.com/developerworks/rational/library/feb06/eeles/
http://www.bredemeyer.com/whatis.htm
http://www.itarchitect.co.uk/articles/display.asp?id=358

Agile, but...

I've heard a lot about the poor effects of "not doing Agile"--that is, doing Agile, but...

These practitioners have a set of expectations around both process and performance. The problem I have is a huge lack of comparative data. When a team is delivering software, it's the rare business that's willing to risk productivity for an experiment to answer questions like "if they can produce x per week under process y, does x decrease if we drop agile practice 'z'?".

When I joined my current project, we were being managed in a barely-recognizable devolvement from scrum. We stopped doing stand-up meetings daily about 4 months ago; we stopped doing status meetings every other day about 8 weeks ago.

Is that negative stuff? I don't have any data either way, but I don't think so. In fact, I think we could argue that we are using exactly the agile things this team needs and can use effectively:

I delivered a formal design document about 2 weeks ago, just in time to start coding for this release cycle. We don't do much formal design, though; we mostly work from a domain model, process specifications, and FitNesse tests. Sometimes, though, you need words-in-a-row and UML diagrams.

We went into code freeze last Friday, limiting check-ins to bug fixes and test fixtures. The following Monday we received several new requirements. Since code freeze, there have been updates to something like 9 process specs. Most, if not all, of those changes will be in the release. We'll probably be in code freeze for two or three weeks, which is a lot in a six-week cycle. We're treating the new requirements as bugs, and to be fair, they're about the size you'd expect from a functionality bug.

We're delivering new functionality at maybe three times the rate of most projects I've been associated with, and at probably 6 times the rate of a typical project in this organization. It's pretty high quality stuff, too, in an environment with hideously complex business rules and very high cost for failure in implementing those rules. Frankly, I'm pretty happy about that.

Here's what we've got that works:

1. very senior, very fast developers. (Plug: if they're from Gorilla Logic, as I am, they couldn't be anything else, as that's a primary focus of Gorilla Logic's hiring practice.)

2. dead quiet when it's time to concentrate. I have a private office. I'm not physically in the same state as the users. In fact, I'm two time zones away. When it's time to crank out well-thought-out results, nothing can beat quiet. (For experimental data on this topic and other key contributors to developer productivity, read "Peopleware" by DeMarco and Lister.)

3. good access to the team when it's time to talk. We have web conferencing, IM, telephones, and desktop video conferencing when we need it.

4. requirements people who aren't bound by the need to generate a document to get the requirements to us, who are highly accessible, and who know the domain quite well.

5. management who understands the importance of getting it right, and respects the professional opinions of the team.

The release will probably slip a week. When we're done, it'll be pretty much what they needed, rather than arbitrarily "what we had working on the release date". We might deliver faster with a different management style, but I wouldn't bet my next paycheck on that. If the problem domain was simple enough to allow for stable requirements, they might not need the software in the first place.

That's agile enough for me.

"Really Good Programmers"

A young designer/developer/businessman I respect and admire twittered the following today:

"Really good ruby programmers can write code that is so clever that no one less talented than them has a hope of understanding it."

He really should know better.

Really good programmers can, indeed, write code so clever that no one less talented than themselves has any hope of understanding it--but they never do. That's the sort of thing really talented coders do before they become really good programmers. And then, in the code review, their peers tell them to take out the clever bit of code and make it maintainable, thereby vastly improving the code.

A decent piece of software has a lifetime measured in tens of years. During that time, it's likely to be looked at a dozen times for new features, bug fixes, and efficiency updates--not to mention the occasional port. If it took a day to write it initially, it'll probably spend many times that under someone's lens while it's in the maintenance phase. Taking 12 hours to write clear, comprehensible code instead of 8 hours to do something clever will save time in the long run. Is it always worth the extra time? No--but it is often enough that I try to do it every single time I code something.

Strange clever code is fun to write. I write a little self-modifying routine one time, both to prove I could do it, and to solve a timing issue--I counted op-codes and found I could squeeze pattern recognition between bytes on a 19.2kbps serial connection by making the polling routine self-modifying. Nowadays I'd comment it like crazy, and advise that it be removed as soon as CPU speeds increased enough to do so. Back then I wasn't a good enough programmer to know I needed to do that. Hadn't looked at enough of my own code, incomprehensible after 5 years of "rest", to know it was important to do so.

So Jon: educate that Ruby programmer of yours. Someday he'll thank you for it. All your maintenance guys will thank you for it, and you won't have to wait for that.

architecture by accident

I’ve been on a number of projects with a common problem set:

• architecture is deteriorating with time, and was never strong to begin with
  
• there is no architect or designer with project-wide authority
  
• nobody views this as a problem–including the development team
  
• time to add a feature is increasing
  
• quality issues continually surface as new features affect older ones
  

The introduction to Booch/Rumbaugh/Jacobson "The Unified Modeling Language User Guide" makes the case for design modeling thusly:

"If you want to build a dog house, you can pretty much start with a pile of lumber, some nails, and a few basic tools...

"If you want to build a house for your family, you can start with a pile of lumber, some nails, and a few basic tools, but it's going to take you a lot longer, and your family will certainly be more demanding than the dog. ...

"If you want to build a high-rise office building, it would be infinitely stupid for you to start with a big pile of lumber, some nails, and a few basic tools. ... You will want to do extensive planning, because the cost of failure is high. ...

"Curiously, a lot of software development organizations start out by wanting to build high rises but approach the problem as if they were knocking out a dog house."

(The full text is entertaining; go forth and read the whole thing.)

As a developer, there are a couple of approaches I use to at least try to address this:

• When adding new features myself, try to talk with everyone with code touching mine. Often there is a conceptual design in each area's designer's mind--there just is no globally-available expression of it.
  
• I document my own designs with diagrams and written discussion, which I pass around for comment. I try to include global information about the global concept objects I touch. This has the effect of (often) flushing out different views of the responsibilities and limitations of key objects.
  
• I transfer design information into javadoc, so the next developer can see it as (s)he codes.
  

Design is key to success in anything but the smallest projects. I use "success" very carefully here; some more recommended reading:

http://thedailywtf.com/Articles/What_Could_Possibly_Be_Worse_Than_Failure_0×3f_.aspx
"When an enterprise software application is deployed to production, it’s almost universally considered to be a success. In most other industries, equating completion and success would be ludicrous. After all, few in the construction industry would deem a two-year-old house with a leaky roof and a shifting foundation anything but a disaster — and an outright liability.

"Yet, had the VCF system gone live, almost everyone involved — from the business analysts to the programmers to the project managers — would have called their work a success. Even after the code devolved into an unsustainable mess, many would still refuse to admit that they had failed. The result is a cycle of failure, as dev shops refuse to recognize and correct issues that plague their projects."

FlexMonkey: test automation for Adobe Flex

I'm a real fan of automated testing. I really don't understand the idea of teams of people driving a UI as a way of regression testing, though there are certainly a lot of major products out there which use exactly that method to test. I think one big reason for that is that user interfaces which aren't written in HTML are notoriously hard to regression test.

A lot of what we do at Gorilla Logic is Flex atop Java. As a result of that, we had to find a way to regression test Flex user interfaces; it's not really practical for an agile team to survive without automated regression testing. Enter FlexMonkey. FlexMonkey was developed and offered to the community in open source as THE way to do regression testing for Flex, and having seen it in action, I have to say I'm really impressed. It's easy to get started with, easy to use, and easy to fully automate. The documentation is enterprise-grade, and best of all, it's free. It took me about half an hour to get it running and automated under ant, and I've never coded in Flex.

So go check it out: FlexMonkey 1.0 released in Google Code (open source) yesterday. http://flexmonkey.gorillalogic.com/gl/stuff.flexmonkey.html.

Here's a bit more about Flex and what we do at Gorilla Logic: http://www.gorillalogic.com/what.development.services.flex.html.