Friday, March 16, 2012

Converting Forms in Restlet to POJOs with Jackson and Guava

Punchline first. This article presents code that you can use to convert HTML forms into POJOs, mapping property names in the natural way, with support for compound objects and arrays/lists. A long-winded set-up for that punchline follows.

Many of my Restlet resources are designed to support both JSON and HTML representations with resource interfaces that look like this:

public interface PersonResource {
    @Get Person getPerson();
}

where Person is a POJO that might look like this:

public class Person {
    public String getName() { ...}
    public List<Address> getAddresses() { ... }
    ... other bean stuff: setters, fields ...
}

and the implementation of the resource looks like this:

public class PersonServerResource 
        extends ServerResource implements PersonResource {
    @Override public Person getPerson() {
        return ... construct Person from domain data ...
    }
}

I register custom ConverterHelpers with the Restlet Engine. One of them knows how to use Jackson to convert Person into JSON, one knows how to obtain a Freemarker HTML template associated with the Person type and render it with the Person as its data model.

It works like magic. When I use a browser to point to the resource, it returns the HTML generated with Freemarker. When I ask for it with, say, jQuery, requesting JSON, I get ... JSON. When I get a Restlet ClientResource proxy for PersonResource at that URI, I get a Person object back when I call getPerson() on the client side; the value returned by getPerson() on the server is serialized to JSON, sent over the wire, and deserialized back into a Person object.

The only problem I had was for PUT and POST method handlers, because there was no automatic way to convert form data to my domain types. I had to create an additional method to handle forms as parameters:

public interface PeopleResource {
    @Post("json:json") Person addPerson(Person person);
    @Post("form:html") Person addPersonForm(Form person);
}

public class PeopleServerResource 
        extends ServerResource implements PeopleResource {


    @Override public Person addPerson(Person person) {
        ...
    }
    @Override Person addPersonForm(Form personForm) {
        Person person = new Person();
        // Extract values from personForm and set
        // them to person.
        return addPerson(person);
    }
}

I suffered this for a long time, and then I began to think that the application/www-form-urlencoded media type was, at least for simple types, not far from JSON. Could I translate form data to JSON and then use Jackson to deserialize that? Of course I could. In fact, it wasn't too hard to define some conventions whereby sequences (lists/arrays) and nested structures could be modeled. An encoded form for Person as produced  by an HTML page might look like this, in part:

name=Tim&address.1.street=Main&address.1.city=Anytown...
     ...&address.2.street=Elm&address.2.city=Smallton...

and the corresponding JSON would be:

{
    "name":"Tim",
    "address":[{
        "street":"Main",
        "city":"Anytown",
        ...
    },{
        "street":"Elm",
        "city":"Smallton",
        ...
    },
    ...
    ]
}

I wrote FormDeserializer to do this. It takes a Jackson ObjectMapper to do the heavy lifting. Here's the Javadoc for the deserialization method:


public  T deserialize(Form source, Class targetType)
Converts a Form to a Java object of the target type, using each name=value pair to set the corresponding property on the target object.

Compound names, using period (.) as the delimiter, are treated as pseudo-dereferences (a la JavaScript or Groovy) to set properties of sub-objects, e.g., a.b=c for bean targets is treated like a call to target.getA().setB(c).

Numeric components of compound names are treated as indices into a sequence named by the preceding components, e.g., a.1=c is treated as target.getA()[1] = c (or target.getA().set(1, c), if the "a" property of the target is a list rather than an array). Unless an element with index 0 is set, the indices are origin 1.

Sequences are also created by names with multiple values, e.g., a=x&a=y is equivalent to a.1=x&a.2=y, with the value of the "a" property in the target being a sequence of two values.
When a name appears both indexed and non-indexed, the last assignment wins: a=x&a.1=a1&a.2=a2 will set the "a" property to a sequence of values [a1, a2], but a.1=a1&a.2=a2&a=xwill set the "a" property to x.

If the top level consists only of integer indices, the subobjects will be interpreted as elements of a sequence (and T must be a List or List subtype instead).

The values are deserialized using the Jackson ObjectMapper that was used to construct this FormDeserializer. Any type that can be deserialized from a string can be used. While it is possible for Jackson to deserialize object graphs that have internal references, it is not possible for the form values to refer outside of themselves.

Runtime exceptions from Jackson conversion are propagated without exception translation. This could be considered a bug.
Parameters:
source - the Restlet Form object to be deserialized
targetType - the type of the target object into which the form is to be deserialized.
Returns:
the deserialized object of the target type


The converter class that uses this machinery is FormConverter. It uses a marker annotation, FormDeserializable, that signals when a class is suitable for deserialization from a form.

The upshot is that I can now write:

public interface PeopleResource {
    @Post Person addPerson(Person person);
}
// And no need for additional method in implementation!


Big reduction in the amount of boilerplate I have to write, and the code is easier to read.

Here are the links to the code in one place:


I have @Inject tags on some constructors, but you can ignore them unless you want to use them for dependency injection.

The implementation makes heavy use of Guava. If you don't or can't use Guava, then you'll have to roll your own machinery. (Good luck!)

6 comments:

Richard Berger said...

Sorry if this is a dupe, posted two days ago but didn't seem to show up...
Tim:

Thank you so much for posting this - it is exactly what I was seeking to solve the same problem you observed. Even though your instructions are very clear, somehow I managed to not get this to work.

After installing the code and guava.jar, I marked the PoJo class as @FormDeserializable and then I removed the following line from the Resource file:
/* @Post("form") Representation postForm(Form form);*/

And I removed the corresponding method from the Server file.

However, now when I run my test, I get

Unsupported Media Type (415) - Unsupported Media Type
at org.restlet.resource.ClientResource.doError(ClientResource.java:582)
at org.restlet.resource.ClientResource.handleInbound(ClientResource.java:1158)
at org.restlet.resource.ClientResource.handle(ClientResource.java:1025)
at org.restlet.resource.ClientResource.handle(ClientResource.java:1000)
at org.restlet.resource.ClientResource.post(ClientResource.java:1409)
at org.restlet.resource.ClientResource.post(ClientResource.java:1355)
at com.fourspires.api.test.BasicCommitmentTests.addResourceDTO(BasicCommitmentTests.java:285)

My test code is essentially...
stateTransitionsClient = ((ClientProxy) stateTransitionsResource).getClientResource();
Form stateTransitionForm = new Form();
representation = stateTransitionsClient.post(form);

Any suggestions for what to investigate? I am using Restlet-GAE 2.2 Snapshot.

Thanks so much!
RB

Tembrel said...

Looks like a client-side problem; it's not even getting to the FormConverter on the server side.

Try setting up a test HTML page with a form whose action is the URI of the target resource and inputs to set the values of the POJO. Then, once you're sure that's working, you can debug the client-side testing piece.

Richard Berger said...

Thanks for the feedback. Here's what I did...

1. I restored my Resource and ResourceServer to their original condition
2. Created HTML page with a form that adds a Resource (State), tested it, works fine
3. I then go to the Resource interface, comment out the "form" method
4. I then go to the ResourceServer class, comment out the "form" implementation
5. Restart server
6. Try HTML page, but now it fails with the 415 error (Unsupported Media Type). As you wrote, I don't seem to even get to the server side.
Note: State and its DataTransferObject (StateDTO) are annotated as @FormDeserializable

I have a feeling that I am making some type of obvious mistake. Trying to understand the flow...
* The webform/client code call post() on the /states/ resource (I switched from /stateTransitions in today's testing)
* That call is routed to StatesServerResource.class, whose interface now contains just the one line:
@Post Representation postJson(StateDTO state);
(Note: I am intending that this is the equivalent of your @Post Person addPerson(Person person); )
* I am guessing that the system should match the postJson method to my form submission, but it appears not to do so and things stop.

If I change:
@Post Representation postJson(StateDTO state);
To:
@Post("form") Representation postJson(StateDTO state);
I get the same 415 error, but now in my server log I find:
Apr 16, 2012 10:58:41 AM org.restlet.service.ConverterService toObject
WARNING: Unable to find a converter for this representation : displayName=formTestDName&phaseName=formPhase&canonicalName=formTestCName

Could the problem be changes made to Restlet in 2.1/2.2 to improve the content negotiation? Or should I be call the FormConverter service explicitly? Thanks again for your help on this! It would be really cool to remove all that duplicate form/object code on the server side for each object.

RB

PS - Awesome blog site here :)

Tembrel said...

Is it possible you aren't registering FormConverter with the Restlet Engine? Something like Engine.getInstance().getRegisteredConverters().add(new FormConverter()) is needed.

If that's not it, I'm stumped.

Richard Berger said...

I am definitely NOT registering the FormConverter (I was wondering what magic was involved in getting the FormConverter to run). I will try that over the next few days and get back to you.

I had read the paragraph that starts "I register custom ConverterHelpers ..." but when I got to Freemarker part, my brain panicked about learning something new and I forgot everything in the paragraph. I am confident that registering the FormConverter will do the trick.

Thanks again!!
RB

Richard Berger said...

Registering the FormConverter made all the difference!

I added the following code to my createInboundRoot() method (wasn't really sure where to put it):
ObjectMapper objectMapper = new ObjectMapper();
FormDeserializer formDeserializer = new FormDeserializer(objectMapper);
Engine.getInstance().getRegisteredConverters().add(new FormConverter(formDeserializer));

and then I was able to simplify the StatesServerResource class. It was like magic. I didn't believe it worked until I ran my test suite for the second time.

Thanks again!!
RB