Sunday, May 29, 2011

Using Guice-ified Jersey in Embedded Jetty

While IoC is a terrific way to promote software modularity, someone still has to write that bootstrap code to connect all the dots. Even though there is abundant information on how to use Guice, Jersey and Jetty respectively in a servlet environment, little is known about using all three together to programmatically configure servlets in Java. Here I will use an example to demonstrate how to write RESTful servlet using all three:
  1. Assemble framework libraries using Maven.
  2. Create a POJO interface and an implementation.
  3. Write a JAX-RS resoure to use Guice constructor injection and JAX-RS annotations.
  4. Write the Guice bootstrap code.
  5. Write the embedded Jetty start up code.
  6. Run and test.
First, assemble all necessary parts in a Maven pom.xml like this:
   
       7.4.1.v20110513
       1.7
       3.0
   

   
    
        org.eclipse.jetty
        jetty-servlet
        ${jetty.version}
    
 
     com.google.inject
     guice
     ${guice.verion}
 
 
     com.sun.jersey
     jersey-server
     ${jersey.version}
 
 
     com.sun.jersey.contribs
     jersey-guice
     ${jersey.version}
  
    
      junit
      junit
      ${junit.version}
      test
    
   
     
  
 
     maven2-repository.java.net
     Java.net Repository for Maven
     http://download.java.net/maven/2/
     default
  
  

We start with a simple POJO interface:
public interface GuicyInterface {
   String get();
}
And a simple implementation:
public class GuicyInterfaceImpl implements GuicyInterface {

   public String get() {
      return GuicyInterfaceImpl.class.getName();
   }
}
Now, write a JAX-RS resource to use both Guice and JAX-RS annotated injections:
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;

import com.google.inject.Inject;@Path("/helloguice")

public class HelloGuice {
   private final GuicyInterface gi;
   
   @Inject
   public HelloGuice(final GuicyInterface gi) {
      this.gi = gi;
   }
   @GET
   @Produces("text/plain")
   public String get(@QueryParam("x") String x) {
      return "Howdy Guice. " + "Injected impl " + gi.toString() + ". Injected query parameter "+ (x != null ? "x = " + x : "x is not injected");
   }
}

Next, compose POJO bindins in a JerseyServletModule. This module will setup the Jersey-based JAX-RS framework for use with Guide injection. The GuiceServletContextListener is used to bootstrap Guice when the servet context is initialized.
import com.google.inject.Guice;
import com.google.inject.Injector;
import com.google.inject.servlet.GuiceServletContextListener;
import com.sun.jersey.guice.JerseyServletModule;
import com.sun.jersey.guice.spi.container.servlet.GuiceContainer;

public class HelloGuiceServletConfig extends GuiceServletContextListener {
   @Override
   protected Injector getInjector() {
      return Guice.createInjector(new JerseyServletModule() {
         @Override
         protected void configureServlets() {
            // Must configure at least one JAX-RS resource or the 
            // server will fail to start.
            bind(HelloGuice.class);
            bind(GuicyInterface.class).to(GuicyInterfaceImpl.class);
            
            // Route all requests through GuiceContainer
            serve("/*").with(GuiceContainer.class);
         }
      });
   }
}

Finally, write the main method using embedded Jersey to start Guice and Jersey together.
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.servlet.DefaultServlet;
import org.eclipse.jetty.servlet.ServletContextHandler;

import com.google.inject.servlet.GuiceFilter;

public class GuiceLauncher {
   public static void main(String[] args) throws Exception {
      // Create the server.
      Server server = new Server(8080);
      
      // Create a servlet context and add the jersey servlet.
      ServletContextHandler sch = new ServletContextHandler(server, "/");
      
      // Add our Guice listener that includes our bindings
      sch.addEventListener(new HelloGuiceServletConfig());
      
      // Then add GuiceFilter and configure the server to 
      // reroute all requests through this filter. 
      sch.addFilter(GuiceFilter.class, "/*", null);
      
      // Must add DefaultServlet for embedded Jetty. 
      // Failing to do this will cause 404 errors.
      // This is not needed if web.xml is used instead.
      sch.addServlet(DefaultServlet.class, "/");
      
      // Start the server
      server.start();
      server.join();
   }
}

If you run the main program on your local host, you can test the servlet using this URL:
http://localhost:8080/helloguice?x=q
Then, you should see a response like this:
Howdy Guice. Injected impl GuicyInterfaceImpl@3aaa3518. Injected query parameter x = q
Congratulations! You have now mastered the three most popular IoC frameworks for programming RESTful servlets!

Thursday, May 5, 2011

Singleton Injection in JAX-RS

JAX-RS defines a Java API for implementing RESTful server-side functions in Java. Being a post-Spring framework, it uses annotations extensively to denotate "resource" injection points. One of these DI annotations is called Provider. A provider is an application supplied class used to extend a JAX-RS run-time such as Jersey. A JAX-RS implementation is required to load only one instance of a provider class in a JAX-RS run-time instance. An application developer can leverage this feature to inject singletons, e.g. a database manager class, into JAX-RS resources. Here is an example on how to do this.

First, define the singleton class.
public class MyDBManager{
   public void store(final String key, final String value) {
      // Store the key-value pair in a DB.
   }
}
As you can see, this class is just a POJO. No special singleton marking anywhere in the class definition.

Next, we will use ContextResolver and @Provider annotatoin to turn MyDBManager into a JAX-RS provider.
import javax.ws.rs.ext.ContextResolver;
import javax.ws.rs.ext.Provider;

@Provider
public class DBResolver implements ContextResolver<MyDBManager> {
   private MyDBManager db;
   
   // A provider must have at least a zero-arg constructor.
   public DBResolver() {
      db = new MyDBManager();
   }
   public MyDBManager getContext(Class type) {
      return db;
   }
}

Now we can inject MyDBManager into a resource class through @Context like this:
import javax.ws.rs.Consumes;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.Context;
import javax.ws.rs.ext.ContextResolver;

@Path("/myresrc")
public class MyResource {
   @POST
   @Path("/{id}")
   @Consumes("text/plain")
   @Produces("text/plain")
   public String post(final @Context ContextResolver myDbManager,
                      final @PathParam("id") String id) {
      myDbManager.getContext(MyResource.class).store(id, "someValue");
      return "OK";
   }
}

Two observations from this method of injecting a singleton:
  • The singleton instance of DBManager is not instantiated until MyResource is invoked the first time.
  • The type T in ContextResolver<T> can be an interface so an implementation of the ContextResolver<T> can return an implementation of the T. But, the implementation of T can only be determined by a Class object.
Therefore, this @Provider-based injection method should not be taken as a general purpose DI mechanism because JAX-RS is not a general purpose DI framework. It is not only ugly but also comes with a reflection overhead when the resource is invoked. Using Guice or Spring instead for general purpose DI.

Wednesday, March 23, 2011

Porting iptables 1.4.10 to Android

(Update 25 Aprile 2011: This port is now in Google Code called iptables4n1).

Introduction 

The Android source comes with iptables 1.3.7. But that distribution is not compatible with Linux kernel 2.6.32 or newer versions. For example, even though iptables 1.3.7 can be built into the Android Emulator and Google Dev Phone 2 (a.k.a. HTC Magic) which are based on Linux kernel 2.6.29, it won't compile with kernel 2.6.32 used in Nexus One and many newer Android devices. The main difference between iptables 1.3.7 and 1.4.x is the emergence of "xtables" in the latter version. Fortunately, Google has done a commendable job in keeping Android in sync with Linux kernel releases. This makes porting a new version of iptables to Android largely an exercise of makefile changes. I will outline the steps here as a service to the open source community.

Steps

Below are steps for porting iptables 1.4.10 to Android/Linux kernel 2.6.32.
  1. Check out the Android source.
  2. Download iptables 1.4.10 source from the Netfilter project.
  3. Go to the checked out Android source and change to $SRC/external/iptables. Delete the content underneath that directory (or make a backup if you want) and then copy the downloaded iptables 1.4.10 content to that directory.
  4. Create a new Android.mk file under $SRC/external/iptables. This is the Android makefile. You can model yours after the one from the original Android source but you need to accommodate naming changes to iptable extensions that went from libiptXXX in 1.3.7 to libxtXXX in 1.4.10.
  5. Change to $SRC/external/iptables/extensions and create a new create_initext4 file there. 
  6. Change back to $SRC/external/iptables and run make. Fix header inclusion issues as needed.
Sample Android.mk

ifneq ($(TARGET_SIMULATOR),true)
  BUILD_IPTABLES := 1
endif
ifeq ($(BUILD_IPTABLES),1)
LOCAL_PATH:= $(call my-dir)
#
# Build libraries
#
# libxtables
include $(CLEAR_VARS)
LOCAL_C_INCLUDES:= \
    $(LOCAL_PATH)/include/ \
    $(KERNEL_HEADERS)
LOCAL_CFLAGS:=-DNO_SHARED_LIBS
LOCAL_CFLAGS+=-DXTABLES_INTERNAL
LOCAL_CFLAGS+=-DIPTABLES_VERSION=\"1.4.10\"
LOCAL_CFLAGS+=-DXTABLES_VERSION=\"1.4.10\" # -DIPT_LIB_DIR=\"$(IPT_LIBDIR)\"
LOCAL_CFLAGS+=-DXTABLES_LIBDIR
LOCAL_SRC_FILES:= \
    xtables.c
LOCAL_MODULE_TAGS:=
LOCAL_MODULE:=libxtables
include $(BUILD_STATIC_LIBRARY)
# libip4tc
include $(CLEAR_VARS)
LOCAL_C_INCLUDES:= \
    $(KERNEL_HEADERS) \
    $(LOCAL_PATH)/include/
LOCAL_CFLAGS:=-DNO_SHARED_LIBS
LOCAL_CFLAGS+=-DXTABLES_INTERNAL
LOCAL_SRC_FILES:= \
    libiptc/libip4tc.c
LOCAL_MODULE_TAGS:=
LOCAL_MODULE:=libip4tc
include $(BUILD_STATIC_LIBRARY)
# libext4
include $(CLEAR_VARS)
LOCAL_MODULE_TAGS:=
LOCAL_MODULE:=libext4
# LOCAL_MODULE_CLASS must be defined before calling $(local-intermediates-dir)
#
LOCAL_MODULE_CLASS := STATIC_LIBRARIES
intermediates := $(call local-intermediates-dir)
LOCAL_C_INCLUDES:= \
    $(LOCAL_PATH)/include/ \
    $(KERNEL_HEADERS) \
    $(intermediates)/extensions/
LOCAL_CFLAGS:=-DNO_SHARED_LIBS
LOCAL_CFLAGS+=-DXTABLES_INTERNAL
LOCAL_CFLAGS+=-D_INIT=$*_init
LOCAL_CFLAGS+=-DIPTABLES_VERSION=\"1.4.10\"
LOCAL_CFLAGS+=-DXTABLES_VERSION=\"1.4.10\"
PF_EXT_SLIB:=ah addrtype ecn 
PF_EXT_SLIB+=icmp #2mark
PF_EXT_SLIB+=realm
PF_EXT_SLIB+=ttl unclean DNAT LOG #DSCP ECN
PF_EXT_SLIB+=MASQUERADE MIRROR NETMAP REDIRECT REJECT #MARK
PF_EXT_SLIB+=SAME SNAT ULOG # TOS TCPMSS TTL
PF_EXT_SLIB+=TAG
EXT_FUNC+=$(foreach T,$(PF_EXT_SLIB),ipt_$(T))
# xtable stuff
NEW_PF_EXT_SLIB:=comment conntrack connmark dscp tcpmss esp
NEW_PF_EXT_SLIB+=hashlimit helper iprange length limit mac multiport
NEW_PF_EXT_SLIB+=owner physdev pkttype policy sctp standard state tcp
NEW_PF_EXT_SLIB+=tos udp CLASSIFY CONNMARK
NEW_PF_EXT_SLIB+=NFQUEUE NOTRACK
EXT_FUNC+=$(foreach N,$(NEW_PF_EXT_SLIB),xt_$(N))
# generated headers
GEN_INITEXT:= $(intermediates)/extensions/gen_initext4.c
$(GEN_INITEXT): PRIVATE_PATH := $(LOCAL_PATH)
$(GEN_INITEXT): PRIVATE_CUSTOM_TOOL = $(PRIVATE_PATH)/extensions/create_initext4 "$(EXT_FUNC)" > $@
$(GEN_INITEXT): PRIVATE_MODULE := $(LOCAL_MODULE)
$(GEN_INITEXT):
    $(transform-generated-source)
$(intermediates)/extensions/initext4.o : $(GEN_INITEXT)
LOCAL_GENERATED_SOURCES:= $(GEN_INITEXT)
LOCAL_SRC_FILES:= \
    $(foreach T,$(PF_EXT_SLIB),extensions/libipt_$(T).c) \
    $(foreach N,$(NEW_PF_EXT_SLIB),extensions/libxt_$(N).c) \
    extensions/initext4.c
LOCAL_STATIC_LIBRARIES := \
    libc
include $(BUILD_STATIC_LIBRARY)
#
# Build iptables
#
include $(CLEAR_VARS)
LOCAL_C_INCLUDES:= \
    $(LOCAL_PATH)/include/ \
    $(KERNEL_HEADERS)
LOCAL_CFLAGS:=-DNO_SHARED_LIBS
LOCAL_CFLAGS+=-DXTABLES_INTERNAL
LOCAL_CFLAGS+=-DIPTABLES_VERSION=\"1.4.10\"
LOCAL_CFLAGS+=-DXTABLES_VERSION=\"1.4.10\" # -DIPT_LIB_DIR=\"$(IPT_LIBDIR)\"
#LOCAL_CFLAGS+=-DIPT_LIB_DIR=\"$(IPT_LIBDIR)\"
LOCAL_SRC_FILES:= \
    iptables.c \
    iptables-standalone.c \
        xshared.c
LOCAL_MODULE_TAGS:=
LOCAL_MODULE:=iptables
LOCAL_STATIC_LIBRARIES := \
    libip4tc \
    libext4  \
        libxtables
include $(BUILD_EXECUTABLE)
endif


Sample create_initext4

#!/bin/sh
echo ""
for i in $1; do
    echo "extern void lib${i}_init(void);";
done;
echo "void init_extensions(void);"
echo "void init_extensions(void) {"
for i in $1; do
    echo "    lib${i}_init();";
done
echo "}"



That is all. Happy hacking!


Sunday, March 20, 2011

Socket and O_NONBLOCK

It is often stated that a socket is created in blocking mode by default. However, not all implementations of the socket calls adhere to that behavior. Specialized operating systems can be particularly tricky when it comes to blocking and non-blocking mode . One should always test the O_NONBLOCK flag on a socket to be sure if it is in the blocking or nonblocking mode operation before proceeding to send/recv. Here are some example c code to demonstrate how to check and manipulate the O_NONBLOCK flag on a streaming socket.

/* !!! Error checking is omited for brevity !!! */
/* Create a streaming socket */
int sock = socket(AF_INET, SOCK_STREAM, 0);

/* Get the file descriptor flag */
int flags = fcntl(sock, F_GETFL, 0);

/* Set O_NONBLOCK if it is set
 * or unset it if it is not
 */
if ((flags & O_NONBLOCK ) == O_NONBLOCK) {
   fcntl(sock, F_SETFL, flags | O_NONBLOCK);  /* Set */
} else {
   fcntl(sock, F_SETFL, flags & (~O_NONBLOCK));  /* Unset */
}

Tuesday, February 22, 2011

How to Check the Version of a Linux Release

The tool uname is usually used to print the OS version on a Unix or Unix-like machine. But it does not always reveal the actual Linux distribution. For example:

myhost:/home/user$ uname -a
myhost:/home/user $ Linux myhost 2.6.32-24-generic-pae #43-Ubuntu SMP Thu Sep 16 15:30:27 UTC 2010 i686 GNU/Linux


If this Linux distribution is LSB compliant, we can use lsb_release to get what we need:

myhost: /home/user$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 10.04.1 LTS
Release:        10.04
Codename:       lucid


Tips in Using Gcore Dump with Java Tools

When analyzing gcore dump from a running JVM process, it is very important to know which Java binary used by gcore to generate the core dump. This is because gcore may not use the Java binary on user's path. If they are different, the user will not be able to attach to the core file to the Java tool in the path. Knowing this will save you hours of frantic head scratching. More information can be found on this discussion about using gcore with jmap.

Another issue is the core file size. A 32-bit Java tool will have problem attaching to a core dump greater than 2GB in size. A workaround is to generate the gcore on a 64-bit JVM and then use the 64-bit Java tool to load the core file.

Wednesday, February 2, 2011

Socket and DataInputStream

While Java provides great I/O abstraction facilities like InputStream and OutputStream, the non-blocking nature of Java NIO often leads to subtle behavior differences in high-level network programming constructs.  The case in point is DataInputStream, which enables an application to parse Java primitive types right out of an input stream.  However, the stream reading methods in the DataInputStream class exhibit two distinct behaviors when the underline stream is a socket.  The following methods are non-blocking:

read(byte[] b)
read(byte[] b, int off, int len)

A non-blocking read means that the first read(byte[] b, int off, int len)may not return the expected len number of bytes.  Therefore, the reader program is usually put into a loop to read repeatedly until the len of bytes are read or the EOF is reached.

The primitive type reading methods in DataInputStream are blocking.  Take readInt() as an example.  That read will block if there are not enough bytes, 4 in this case, in the stream.

Consistency is a good software engineering principle to practice.  When using DataInputStream to parse a socket input stream, it is best to always use the primitive type reading methods to avoid confusion of blocking vs non-blocking socket I/O.