Converting Forms in Restlet to POJOs with Jackson and Guava

Punchline first. This article presents code that you can use to convert HTML forms into POJOs, mapping property names in the natural way, with support for compound objects and arrays/lists. A long-winded set-up for that punchline follows.

Many of my Restlet resources are designed to support both JSON and HTML representations with resource interfaces that look like this:

public interface PersonResource {
    @Get Person getPerson();
}

where Person is a POJO that might look like this:

public class Person {
    public String getName() { ...}
    public List<Address> getAddresses() { ... }
    ... other bean stuff: setters, fields ...
}

and the implementation of the resource looks like this:

public class PersonServerResource 
        extends ServerResource implements PersonResource {
    @Override public Person getPerson() {
        return ... construct Person from domain data ...
    }
}

I register custom ConverterHelpers with the Restlet Engine. One of them knows how to use Jackson to convert Person into JSON, one knows how to obtain a Freemarker HTML template associated with the Person type and render it with the Person as its data model.

It works like magic. When I use a browser to point to the resource, it returns the HTML generated with Freemarker. When I ask for it with, say, jQuery, requesting JSON, I get ... JSON. When I get a Restlet ClientResource proxy for PersonResource at that URI, I get a Person object back when I call getPerson() on the client side; the value returned by getPerson() on the server is serialized to JSON, sent over the wire, and deserialized back into a Person object.

The only problem I had was for PUT and POST method handlers, because there was no automatic way to convert form data to my domain types. I had to create an additional method to handle forms as parameters:

public interface PeopleResource {
    @Post("json:json") Person addPerson(Person person);
    @Post("form:html") Person addPersonForm(Form person);
}

public class PeopleServerResource 
        extends ServerResource implements PeopleResource {


    @Override public Person addPerson(Person person) {
        ...
    }
    @Override Person addPersonForm(Form personForm) {
        Person person = new Person();
        // Extract values from personForm and set
        // them to person.
        return addPerson(person);
    }
}

I suffered this for a long time, and then I began to think that the application/www-form-urlencoded media type was, at least for simple types, not far from JSON. Could I translate form data to JSON and then use Jackson to deserialize that? Of course I could. In fact, it wasn't too hard to define some conventions whereby sequences (lists/arrays) and nested structures could be modeled. An encoded form for Person as produced  by an HTML page might look like this, in part:

name=Tim&address.1.street=Main&address.1.city=Anytown...
     ...&address.2.street=Elm&address.2.city=Smallton...

and the corresponding JSON would be:

{
    "name":"Tim",
    "address":[{
        "street":"Main",
        "city":"Anytown",
        ...
    },{
        "street":"Elm",
        "city":"Smallton",
        ...
    },
    ...
    ]
}

I wrote FormDeserializer to do this. It takes a Jackson ObjectMapper to do the heavy lifting. Here's the Javadoc for the deserialization method:

public  T deserialize(Form source, Class targetType)

Converts a Form to a Java object of the target type, using each name=value pair to set the corresponding property on the target object.

Compound names, using period (.) as the delimiter, are treated as pseudo-dereferences (a la JavaScript or Groovy) to set properties of sub-objects, e.g., a.b=c for bean targets is treated like a call to target.getA().setB(c).

Numeric components of compound names are treated as indices into a sequence named by the preceding components, e.g., a.1=c is treated as target.getA()[1] = c (or target.getA().set(1, c), if the "a" property of the target is a list rather than an array). Unless an element with index 0 is set, the indices are origin 1.

Sequences are also created by names with multiple values, e.g., a=x&a=y is equivalent to a.1=x&a.2=y, with the value of the "a" property in the target being a sequence of two values.
When a name appears both indexed and non-indexed, the last assignment wins: a=x&a.1=a1&a.2=a2 will set the "a" property to a sequence of values [a1, a2], but a.1=a1&a.2=a2&a=xwill set the "a" property to x.

If the top level consists only of integer indices, the subobjects will be interpreted as elements of a sequence (and T must be a List or List subtype instead).

The values are deserialized using the Jackson ObjectMapper that was used to construct this FormDeserializer. Any type that can be deserialized from a string can be used. While it is possible for Jackson to deserialize object graphs that have internal references, it is not possible for the form values to refer outside of themselves.

Runtime exceptions from Jackson conversion are propagated without exception translation. This could be considered a bug.

Parameters:

source - the Restlet Form object to be deserialized

targetType - the type of the target object into which the form is to be deserialized.

Returns:

the deserialized object of the target type

The converter class that uses this machinery is FormConverter. It uses a marker annotation, FormDeserializable, that signals when a class is suitable for deserialization from a form.

The upshot is that I can now write:

public interface PeopleResource {
    @Post Person addPerson(Person person);
}
// And no need for additional method in implementation!


Big reduction in the amount of boilerplate I have to write, and the code is easier to read.

Here are the links to the code in one place:

• FormDeserializer.java
• FormConverter.java
• FormDeserializable.java

I have @Inject tags on some constructors, but you can ignore them unless you want to use them for dependency injection.

The implementation makes heavy use of Guava. If you don't or can't use Guava, then you'll have to roll your own machinery. (Good luck!)

Restlet Guice extension considered ... unnecessary

I wrote some code back in 2008 to help inject my Restlet resources using Guice and then blogged about it. In 2009, the Restlet team added it to their "incubator" as a potential Restlet extension. (Calling it a Guice extension is a bit of a misnomer, since part of it is independent of Guice and applies to any JSR-330-compliant DI framework.) They've talked about promoting it out of the incubator as early as release 2.2. Update: It's now part of Restlet 2.2 and I've updated code references to point to the 2.2 branch.

More recently, I argued (and Restlet's Jérôme Louvel seemed to accept the argument) that Restlet needs a uniform way to create a non-standard Finder for all the Restlet classes that have methods accepting a Class<? extends ServerResource> in place of a Restlet. This would make the use of the Guice extension invisible when wiring up the routing structure: Currently you have to convert the resource type to a Restlet explicity,

    router.attach("/path/to/resource", finderFactory.finder(MyResource.class));

and the extra support would let you write:

    router.attach("/path/to/resource", MyResource.class);

It's all very gratifying, except I realized suddenly this week that for most people who just want to inject their ServerResource subclasses it's completely unnecessary. It's much simpler to define doInit() to inject the resource's members, and then you can just use the second, simpler form above. I wrote an abstract base class extending ServerResource that does just that:

SelfInjectingServerResource.java

Notice that SelfInjectingServerResource uses only the standard @Inject annotation and nothing specific to Guice. In order to make it actually work with Guice, you need to tell Guice that you want self-injecting server resources. The following class is a Guice Module that accomplishes this by requesting static injection of the SelfInjectingServerResource class and providing an implementation of its nested MembersInjector interface to perform the injection.

SelfInjectingServerResourceModule.java

Install this module when creating the top-level injector, make sure that the server resources you want injected inherit from SelfInjectingServerResource, and it just works. Here's a JUnit test class that shows a trivial example of the idea in practice:

SelfInjectingServerResourceModuleTest.java

We'd like this technique to co-exist with other techniques that manage resource injection differently (e.g., the current Guice extension), without worrying about whether it's OK to extend SelfInjectingServerResource, so this code uses an atomic boolean to prevent multiple injections. (AtomicBoolean is probably overkill, but it can't hurt to be safe.)

[Deleted obsolete discussion of how to use prior to adoption as part of Restlet.]

I said that the old Guice extension was unnecessary for most users, but there are a few things the old code does that the new code doesn't do:

1. It does constructor injection, letting you have final fields with injected values.
2. It lets you create a Finder for a resource interface, decoupling the target of the routing from the resource implementation.
3. It lets you use qualifiers when looking up resource types, further decoupling the target of an attachment from the implementation.
4. It doesn't require inheritance from a common abstract base class.
5. You don't have to remember to call super.doInit() when you override doInit

As it happens, I'm taking advantage of #2 in my current work. (Note that there's no way to avoid the explicit finder call in this case, even with added Restlet support, because the Restlet signatures expect a ServerResource subclass.) So I'm stuck with my mistake for the foreseeable future. But no one else need be, now.

Update (2013 July 9)

I got rid of my dependency on #2 above, and I am now using the new approach exclusively, and I strongly recommend others do the same.

Further update (2014 Sep 1)

The Restlet Guice extension package docs discuss how to use each of three approaches, so you aren't forced to use the approach that I prefer.

Deploying Restlet components in Elastic Beanstalk

I spent some time finding a way to deploy a Restlet component in ElasticBeanstalk without giving up the option of deploying it as a standard Java executable for local development and testing. People have expressed interest in seeing the approach, but I don't have the time to create a full-fledged framework, so I've extracted the following template instead (links to code embedded in the prose). Just copy to your own environment, changing package and class names as you see fit, add your Guice Modules, define your Restlet applications and resources, and you should be able to run the resulting component both standalone and (by bundling everything in a WAR) via Elastic Beanstalk. I make no claims that this code is correct, that it is safe to use, or even that it will compile. You have to read it, compile it, and judge for yourself.

The code depends on Restlet, Guice, Rocoto, Guava, and (minimally) SLF4J. I've included commented-out code that uses the Restlet-Guice extension, which is in incubator status in the Restlet codebase.

The Main class is a GuiceServletContextListener and it has a main method. When you run com.example.server.Main from the command line, the GuiceServletContextListener methods are ignored. When you deploy a WAR that contains the associated web.xml file, the main routine is never called. You can inject the current DeploymentMode into your classes if you want to change behavior depending whether you're running standalone or in Elastic Beanstalk (i.e., as a servlet).

The MainComponent class is where your Restlet component logic goes. Its lifecycle is managed by the MainService class, which is where you can manage other services with lifecycles. (If you use JClouds BlobStores, for example, you can ensure that a singleton BlobStoreContext is closed when the MainService is stopped. Another example: a LifecycleService for the Hazelcast data grid) There is some trickiness to handling shutdown in various cases; make sure you nest try-finally clauses so that every one of your services gets a chance to shut down even if the previous shutdown attempt throws an exception or error.

The MainServletModule and MainServlet classes deal with the details of embedding the component in a servlet (which is necessary in order to use Elastic Beanstalk).

AwsCredentialsModule is an example of how you can inject AWS credentials from either of two sources: system properties defined on the command line (or via Ant invocation) and system properties defined by an Elastic Beanstalk configuration.

The flow of control when running standalone is that the main routine creates an instance of Main with the STANDALONE deployment mode, creates the injector, gets the singleton MainService, and starts it, which causes (in a different thread), the MainComponent to be started.

The flow of control when running as a servlet is that a default instance of Main is created (due to the web.xml directive) and gets the SERVLET deployment mode. Its contextInitialized call creates the injector, which injects the MainService and starts it, causing (in a different thread), the MainComponent to be started. 

Files:

• web.xml - web application descriptor
• Main - has main() routine for standalone; is context listener for servlet mode
• MainComponent - the Restlet component we want to be able to deploy in both modes
• MainService - manages lifecycle of MainComponent and (optionally) other services
• MainServletModule - included in module list in Main; configures Restlet-Servlet bridge
• MainServlet - injects MainComponent and returns it as the component it wraps
• DeploymentMode - enum with two values: STANDALONE and SERVLET
• AwsCredentialsModule - example of passing AWS credentials for binding with @Named

Dependency injection in Restlet 2.0 with Guice

I'm very excited about Restlet 2.0, which is in testing now and scheduled for release at the end of 2009. Among other things, the newly refactored Resource API encourages more readable code through the use of a small number of annotations. Here's a very simple server-side resource:

public class DefaultResource extends ServerResource {
    @Get public String represent() {
        return "Default resource, try /hello/resource or /hello/handler";
    }
}

ServerResource has a no-arg constructor, so you don't need to pass Context, Request, and Response objects to super().

Nice as this is, I still want to inject my resources using Guice, so I've updated the Restlet-Guice classes from my previous post to support both the new ServerResources and the old Handlers. It incorporates the enhancement mentioned in the updates: You can override methods to use different providers for Application, Context, Request, and Response, although for most purposes the default providers should suffice.

Update 2009-Dec-11

I've been working on a Guice extension for Restlet as an incubator project that uses the ideas in this posting. My goal is to make this extension independent of Guice at the interface level, so that people can provide implementations for different DI frameworks, of which Guice is only the first.

Resource dependency injection in Restlet via Guice

(See a more recent posting on this subject.)

I have a body of code that already benefits from Guice dependency injection, and I want to migrate it from a servlet-based architecture to Restlet without losing those benefits. I've had some success, and I figured I'd report on it.

I don't mean that I wanted to use Guice to wire up an object graph of Restlets (Components, VirtualHosts, Applications, etc.). It's straightforward enough to use the builder-like API of Restlet, and such code is nicely confined in a few places where it doesn't bother the meat of the application, its resources.

What I do want is to have those resources created by Guice, so that they can be constructed with dependencies injected into them. With the Finder approach, resource constructors get a Context, a Request, and a Response, but that's it. I suppose I could find a way to stash an Injector in one of these so that resources could look up the dependencies they need, but I wanted something more direct. I came up with the following scheme, which seems to be working pretty well:

I use a custom Finder that knows how to look up a Handler/Resource by class or Guice key -- instances of these custom Finders are provided by a FinderFactory:

public interface FinderFactory {
    Finder finderFor(Key<? extends Handler> key);
    Finder finderFor(Class<? extends Handler> cls);
}

The class RestletGuice has static methods named createInjector that parallel those of the Guice class. The difference is that it adds bindings for Context, Request, Response, and FinderFactory. In my Guiced version of the FirstStepsApplication from restlet.org, the following lines create an Injector with some bindings and then look up the FinderFactory that will produce Finders that will be able to make use of those bindings. (These bindings are not necessarily good practice; they just demonstrate the technique.)

    Injector injector = RestletGuice.createInjector(new AbstractModule() {
        public void configure() {
            bind(Handler.class)
                .annotatedWith(HelloWorld.class)
                .to(HelloWorldResource.class);
            bindConstant()
                .annotatedWith(named(HelloWorldResource.HELLO_MSG))
                .to("Hello, Restlet-Guice!");
        }
    });
    FinderFactory factory = injector.getInstance(FinderFactory.class);

HelloWorldResource is slightly changed. It has a private final field, msg, that is used to generate the text representation. The msg field is initialized via an injected value.

    @Inject public HelloWorldResource(@Named(HELLO_MSG) String msg,
                                      Request request,
                                      Response response,
                                      Context context) {
        super(context, request, response);
        this.msg = msg;
        getVariants().add(new Variant(MediaType.TEXT_PLAIN));
    }
    static final String HELLO_MSG = "hello.message";

This all comes together in the Application class when attaching a Finder for the default routing.

    Finder finder = factory.finderFor(Key.get(Handler.class, HelloWorld.class));
    Router router = new Router(getContext());
    router.attachDefault(finder);
    return router;

I read this as "Route any request for this application to whatever resource is bound to @HelloWorld-annotated Handlers, injecting that resource's dependencies," which was exactly what I wanted.

For those who don't want to have to call a special RestletGuice method, there is a public class FinderFactoryModule extending AbstractModule that can be used with Guice.createInjector(...) to get the same effect.

FinderFactoryModule also implements FinderFactory, so you can construct an instance in your Restlet wiring code and use it right away. You can then use the FinderFactoryModule to create an injector explicitly.

As a special convenience, if you don't use a FinderFactoryModule to create an injector, one will be created implicitly the first time one of its Finders is used to find a target Handler/Resource. This encourages a style where each Application gets its own implicit Injector by creating a local FinderFactoryModule and using it to create Finders.

That's basically it. It works with plain Guice 1.0 and Restlet 1.1 (since it uses Handler as the base type for all Resources). I haven't had time to package it nicely, but you can follow the links below for the actual code.

Sources (links are out of date, use this instead):
FinderFactory.java
FinderFactoryModule.java
RestletGuice.java

Example (links are out of date, use this instead):
FirstStepsApplication.java
HelloWorld.java
HelloWorldResource.java
Main.java

Update (August 2008)

Chris Lee discovered that Context.getCurrent() doesn't work reliably as of 1.1m5; the workaround is to use Application.getCurrent().getContext(). A more comprehensive fix, that I hope to post soon, would involve letting subclasses of FinderFactoryModule override the methods that create the Providers for Request, Response, and Context.

Update (2009-Feb-3)

Leigh Klotz has a workaround for using Guice Finders with WadlApplication. And I still haven't had time to package any of this more nicely. (Feb 7: Jérôme Louvel took Leigh's suggestion and checked it in on the Restlet trunk.)

Concurrency issues in Restlet

Issue #368 in the Restlet issue tracker was originally focused on the Guard class. Since then, I've taken a look at the rest of the org.restlet package and found that the fields of most classes are accessed without appropriate synchronization. [Update 2008-Feb-6: All of these have been fixed as of Restlet 1.1 M1.]

Most of these problems would disappear if the fields could be final, but that would mean taking away setter-injectability.

There is hope, however: Methods that are used only for setter injection (and not, for example, used to reconfigure an instance dynamically), should be documented as setter-injection-only, meaning they must not be called after the instance has been (safely) published. Fields set by such methods get their values before publication, and those values never change; such fields are effectively immutable. I don't know of any standard documentation conventions for this, but the important thing is to decide for each non-final, non-volatile field whether it is truly mutable or only setter-injected before publication.

I have a long-standing gripe about the Spring documentation that it doesn't state, at least not in terms that I recognize, under what conditions beans are safely published. The vanilla use cases (standard ApplicationContext implementations) are almost certainly fine, but if you have any doubts about whether your dependency injection framework can guarantee safe publication, then you should either guard your setter-injected fields with a lock or make them volatile. I have a personal preference for using volatile in this case, but most of my Java Concurrency in Practice co-authors have the opposite preference.

Even if you're fairly confident about your DI framework's guarantees, it is never wrong to guard field access or make a field volatile, only potentially wasteful. For a framework like Restlet that should be usable in any DI context, I think it would be prudent to assume the worst of that context.

But is setter injection really necessary? Most of the injected fields are of distinct types, so constructor injection in Spring would be straightforward. Getting rid of setter injection entirely would allow most, if not all, fields to be final, which would greatly reduce the risk of concurrency problems in Restlet.

Concurrency discussion in Restlet community

I started a thread on concurrency issues in the Restlet framework, and it led to a several people saying nice things about Java Concurrency in Practice.

Jérôme Louvel's responsiveness to these issues is gratifying. A far cry from the huffiness I provoked when I had the temerity to question the concurrency guarantees in Spring!