Appcelerator Titanium KrollDict fails to sanitize JSONObject.NULL values

If you’re developing an Appcelerator Titanium module for Android it’s possible that you’ll run into this exception (see bottom of post for full exception trace):

!!! Unable to convert unknown Java object class 'org.json.JSONObject$1' to Js value !!!

It took me a long time to track down the cause of this error, but essentially the issue is in the JNI layer, which passes native Java objects to the Javascript layer does not know how to handle a JSONObject.NULL value. The Titanium wrapper object KrollDict attempts to convert all non-native Java objects into native ones. JSONObject becomes a simple Map, JSONArray becomes an ArrayList and so on. However, the constructor fails to account for instances of the JSONObject.NULL object, which is not a simple primitive.

An easy work around would be to ensure the JSON objects that you’re passing around never have any true JSON null values in them. You can replace them with empty strings, or simply remove the key from the object altogether. This is ultimately the fix we went with. However, a simple two-line patch should solve this issue in the core Titanium code:

From 57868ea5b822468563d4a1be2adb901cce383f11 Mon Sep 17 00:00:00 2001
From: Tristan Waddington <tristan.waddington@gmail.com>
Date: Mon, 4 Jun 2012 14:24:15 -0700
Subject: [PATCH] Update the 'fromJSON' method of 'KrollDict' to map the
 'JSONObject.NULL' Object to a real Java null
 representation.
 
---
 android/titanium/src/java/org/appcelerator/kroll/KrollDict.java |    2 ++
 1 file changed, 2 insertions(+)
 
diff --git a/android/titanium/src/java/org/appcelerator/kroll/KrollDict.java b/android/titanium/src/java/org/appcelerator/kroll/KrollDict.java
index da29217..96017d6 100644
--- a/android/titanium/src/java/org/appcelerator/kroll/KrollDict.java
+++ b/android/titanium/src/java/org/appcelerator/kroll/KrollDict.java
@@ -56,6 +56,8 @@ public static Object fromJSON(Object value) {
 					values[i] = fromJSON(array.get(i));
 				}
 				return values;
+			} else if (value == JSONObject.NULL) {
+				return null;
 			}
 		} catch (JSONException e) {
 			Log.e(TAG, "Error parsing JSON", e);
-- 
1.7.10

I submitted a pull request to the titanium_mobile repository on GitHub, but all contributors are required to sign a Contributor License Agreement (CLA). Not a big deal, but I don’t expect to ever submit another patch to this project, so I’m not inclined to spend a day getting their dev environment bootstrapped and sign my life away for a two-line bug fix (yes, I wrote a passive-aggressive blog post instead).

Unfortunately, they don’t even seem interested in accepting the patch as a simple bug report so someone else can fix it:

If you’re trying to build a platform that’s meant to be developer focused, willful disinterest is not the best response. If you’re curious, here’s the raw patch file and the full exception:

Add mercurial or git changeset id to your Android app

We recently automated the debug builds of our Android app. As a result, we wanted to include the Mercurial changeset id in our app settings, so we could quickly tell what version of the app someone was running.

What we ended up doing was writing the changset id to a custom properties file that was then copied into the raw directory during a build. That properties file could then be read by our PreferencesActivity on runtime.

In order to do this we wrote a custom build.xml file for ant. Simply copy and paste the following into your build.xml file:

    <!-- Require the hg.revision task during pre-build -->
    <target name="-pre-build" depends="hg.revision" />
 
    <!-- Check to see if a mercurial repository exists in the source dir -->
    <available file=".hg" type="dir" property="hg.present" />
 
    <!-- Get the mercurial changeset id for tip -->
    <target name="hg.revision" description="Store mercurial revision in ${repository.version}" if="hg.present">
        <exec executable="hg" outputproperty="hg.revision" failifexecutionfails="false" errorproperty="">
            <arg value="id" />
            <arg value="-i" />
            <arg value="-n" />
            <arg value="-r" />
            <arg value="tip" />
        </exec>
        <echo message="Repository version is ${hg.revision}" />
 
        <!-- Create property file containing mercurial changeset id -->
        <propertyfile file="version.properties" comment="The changset id that this app was built from.">
            <entry key="changeset" value="${hg.revision}" />
        </propertyfile>
 
        <!-- Move property file to app accessible res/raw/ directory -->
        <move file="version.properties" todir="res/raw/" />
    </target>

Essentially this creates an executable task that calls hg id -i -n -r tip in the build workspace. It then adds the result of that command to a propertyfile called version.properties. That file is then copied to the res/raw/ directory.

Our app code is even more straightforward. We simply wrote a function to retrieve the changeset id from the properties file in the raw resources directory. We then take the output of this function and update the preference item.

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        addPreferencesFromResource(R.xml.settings);
 
        // Display app_build in settings as set in the app.version properties file at build time
        Preference build = findPreference("app_build");
        build.setSummary(this.getAppChangsetFromPropertiesFile());
    }
 
    public String getAppChangsetFromPropertiesFile() {
        Resources resources = getResources();
 
        try {
            InputStream rawResource = resources.openRawResource(R.raw.version);
            Properties properties = new Properties();
            properties.load(rawResource);
            return properties.getProperty("changeset");
        } catch (IOException e) {
            Log.e(TAG, "Cannot load app version properties file", e);
        }
 
        return null;
    }

Android app build automation with Jenkins

In a nutshell, Jenkins provides an easy-to-use so-called continuous integration system, making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build.

This post assumes you have a working ci server running Jenkins (or Hudson).

First download and install the Android SDK to your Hudson server. Make sure it’s in a directory that’s accessible by your Hudson user.

$ sudo su hudson
$ cd ~/
$ curl -lO http://dl.google.com/android/android-sdk_r11-mac_x86.zip
$ unzip android-sdk_r11-mac_x86.zip

After unpacking the SDK you’ll need to install the individual Android platforms.

$ cd android-sdk-mac_x86/
$ android update sdk --no-ui

Now go get some coffee or take a smoke break, this is going to take a while.

Phew. Okay, now that’s done we can configure our new Hudson task. I’m going to assume you’re somewhat familiar with boot strapping a new project in Hudson, so I’m going to gloss over some of the details. Just make sure it’s pointed at your Android app’s source code repository.

In the build section of the project config you’ll want to specify a new “Execute Shell” build step with the following script.

# Ensure the SDK is in the Hudson user's system path
PATH=/Users/hudson/android-sdk-mac_x86/tools/:$PATH
 
# Change to the Hudson workspace directory
cd $WORKSPACE
 
# Create the required build files in the workspace
android --verbose update project --path .
 
# Execute the build
ant clean debug
 
# Copy the apk out of the workspace so your testers can get at your fresh build
scp bin/YourAppName-debug.apk foo:bar/

Pro Tip: Add this build script to your code repository and simply execute that script from Hudson. That way you can track your changes.

Note that if you’ve bundled extra libraries into your app you may see a build error when running this code. If so, try executing the ant build with the -lib option like so.

$ ant clean debug -lib foo/bar/libdir/

If you get into trouble, know that you can execute this build from the console as the Hudson user. Try running through the script manually first so you can identify any errors.

Update (2012-02-01): After living with this set up for a few months we started to run into some issues. It turns out, running your Ant builds from a shell script means Jenkins won’t always notice that your builds are failing. We also started running JUnit tests that, when a test failed, did not also trigger a build failure.

As a result we discovered that it was far better to run your Android builds using the Jenkin’s Ant Plugin. This allows you to invoke a series of Ant commands like clean debug or clean release. Jenkins will also format the Ant output and properly mark your build as failed or successful.

You can also take advantage of the Post Build Task plugin to run a script after your build is successful. This is useful for copying your .apk files elsewhere.

Don’t forget to run $ android update project --path . before your Ant commands are invoked, or your Android SDK folder may not be properly linked to your project.

Update (2012-09-06): The Android Emulator Plugin for Jenkins is a great tool that will help you manage the Android SDK on your CI server. It’s a far better option than installing the tools yourself. While you’re at it, you should check out the Android Lint Plugin to make sure your project is in tip-top shape.

Update (2012-03-02): I recently did a talk on Android build automation with Ant and Jenkins at the 2013 Snow-Mobile conference in Madison, Wisconsin. You can view the slides from the talk online at docs.google.com/presentation/d/19ddXSEVfd6-bGItNfROJMlL6O2rA1XOcRiBdE3NpgV4/edit?usp=sharing.