Of course, if the bad guys don’t know how the processing of input is implemented, it will be tricky for them to find the hole to exploit. So I suppose blocking the specific method (as Tomcat did by limiting the number of parameters it will hash) serves to block opportunistic attacks. But it may still leave possibilities for those who are really determined to cause havoc with a specific site.

1. Facebook – every link that leads to the browser wastes my time going through some stupid Facebook warning *in the browser* that links can, you know, lead to unpredictable stuff, so be careful. Something that if it were necessary at all, would make a lot more sense on a PC (where no such warning is given) than a mobile device IMHO, and did I mention, wastes my freaking time and can’t be disabled as far as I can see.
2. The email app is atrocious. Not GMail, talking about what you use for e.g. an Exchange account. Has only rudimentary understanding of folders, and no way of getting notifications only for specific folders. Wonder if there’s a better app for that.

I also had a new idea for an app (probably already implemented somewhere) – hotspot fast track. When you’re at a public hotspot (or e.g. one at a hotel), every time you connect it wants you to go to some BS web page (usually requiring BS JavaScript) and click through some BS agreement to actually use the internet. As if everyone only ever used the web and anyone gave a shit about the stupid agreement. When you’re on a mobile this is a huge PITA, not least because you don’t tend to maintain a steady connection, and find yourself going back at that BS web page once you notice the network doesn’t seem to be working. The hotspot fast track app would look for wifi connections, test for BS portal redirections, and automatically “click through” known portal types that you authorize.

My project in 1994 (at the NSF Geometry Center in Minneapolis) centered around a whimsical topic, soap bubbles (or more generally, “thin films”). I wanted to find a way to render them, in particular their swirling iridescent colors, which took me on a foray into understanding their physics. As part of my summer project there, I created a rudimentary ray tracer (more of a proof of concept than anything; I didn’t know anything about ray tracers when I started) just for the purpose of rendering thin films with some degree of realism. Unfortunately the Geometry Center and any records of my project are all long gone now, though I still have a poster and a video I created, perhaps some code somewhere, and a vague memory of the physics.

The reason thin films are colored has to do with interference effects of light; the film is so thin (on the order of the wavelengths of visible light) that reflections of a coherent light ray from the inner surfaces of the film interfere with the original ray, causing that wavelength to come out lighter or darker depending on the exact thickness of the film versus the wavelength. The physics are explained reasonably well at Wikipedia. The variations in color across the film are due to slight variations in thickness (resulting from gravity, evaporation, wind, viscosity, etc.) which cause different wavelengths of light to interfere more or less.

The point is that this is happening at the wavelength level. Now imagine a naive computer graphics approach to rendering this; you would calculate the effect on red, green, and blue light – three wavelengths – and that would be your color. But it should be obvious that three wavelengths hardly represent the whole spectrum, at least when the effects are occurring at the level of individual wavelengths. And indeed, the results were pretty unconvincing. For a better approximation, I added three wavelengths, although I can’t remember now how that worked, but at least it was easy to work into a color model, although it may have been wrong-headed (see for reference Magenta Ain’t A Colour). The color bands did look better. Wish I still had the pictures from then to show here.

In 1996, as part of a computer graphics project course at NCSU, I revisited this problem with the idea of using the full spectrum. In an ideal world, we would model a ray of light as a continuous spectral function I(lambda) – intensity at each wavelength – and interactions with objects (such as reflection) similarly as functions Rn(lambda) to transform brightness of the light at each wavelength, with the result being simply the product of all interaction functions Rn and the original ray of light I. The problem is, what exactly does this look like? We don’t perceive a spectrum of light; we perceive a single color. How do I translate a wavelength function into an RGB value to display on a computer screen?

In 1996, the web did not extend (as far as I could find – actually it may have) to a deep practical treatment of how to do this. There may have been articles like this one now on Wikipedia to explain the concept fairly clearly, but for the actual empirical numbers I had to dig up an obscure book in the library. To summarize, our eyes have color receptors of three different varieties, each of which is stimulated differently by a different range of visible wavelengths – one mostly by longer wavelengths, one mostly by shorter wavelengths, and one mostly by the mid-range. In fact each could be modeled by a stimulation function Sreceptor(lambda) which could be multiplied by a spectral function I(lambda) and integrated over the visible spectrum to determine that receptor’s stimulation by a particular ray of light (see that Wikipedia article).

These functions are my key to mapping a spectral intensity function into values I can display. In the 1930s a set of experiments (I can only imagine as incredibly tedious) mapped typical receptor responses as a function of various wavelengths. The receptiveness of the three receptors defines a three-dimensional space called (somewhat arbitrarily) the CIE XYZ color space. Red (long), blue (short), and green (midrange) light can be used to stimulate the individual receptors almost in isolation, which is why monitors only need to emit three wavelengths of light to cover most of our range of color perception.

But that doesn’t mean that these three wavelengths are representative of interactions with the entire spectrum, and thin films are a clear example of the inadequacy. Using the forty or so wavelengths sampled in my sources, I got very convincing renderings indeed. My film thickness models were trivial -  just linearly decreasing from bottom to top – but the bands of color looked very realistic.

Everyone’s familiar with light sources having different spectra. This is why everything looks amazing in department store lights, and why your digital camera likely has different settings for light sources that are incandescent, fluorescent, sunny, cloudy, etc. We can easily see these effects, yet the standard RGB representation of color contains no hint that more than three wavelengths are involved. We know that color interactions actually occur due to the surface characteristics at the microscopic (indeed, molecular) level where all different wavelengths must be affected differently. So why not treat them that way in computer graphics?

My hope is to integrate my full-spectrum  model into an existing renderer, not only allowing me to render films along with normal scenes, but also to apply texture maps to films as thickness maps, thereby modeling the variations seen in actual films (how cool would it be to write your name on a bubble this way?). An open source project like POVray would be perfect. When I looked at this program in particular, I was hoping that color would be represented by a class that I could simply modify. Unfortunately the implementation was more integrated – macro-based, as I recall – probably for performance reasons. Perhaps a different program would be more amenable. But in any case, I would run into the problem of representation: no one is used to specifying colors as a sampling at 40+ wavelengths. Everyone’s scene definitions use RGB colors, and there’s no unique mapping of an RGB coordinate into such a representation (which when you think about it is the whole point here – two surfaces may both appear the same under one light source, but reflect very different colors under another, due to different interaction functions) – in fact there are infinitely many. In order to specify realistic colors as a full spectrum, we’d have to go around measuring in the real world with a spectrometer.

Still, this is an intriguing idea to me, and I’d really like a chance to see how it works out one day.

Although I understand that Google has had to contend with both the open source zealots and the closed-everything carriers, upon looking at the trend, I find Google’s actions getting more disturbing. Just as Android seems to be coming into its own and Google should have more power than ever to twist arms, Google seems to be wimping out – or turning evil. I hope I’m wrong and they’re just waiting for the right time.

One thing I completely forgot to talk about is the abandoning of the Nexus One. When it came out, it was supposed to herald a new age of cross-carrier, stock-Android phones (with a built-in connection-sharing capability, no less). Only T-Mobile really picked it up – you could use it on AT&T but without 3G. Verizon and Sprint were supposed to be coming out with support for the same concept and just a different radio, but instead they released their own phones, with the usual modifications and constraints. So why did Google let them? They didn’t have to; the Skyhook case shows that Google can essentially pull their blessing from any phone for any reason. An Android phone without the Google apps isn’t going to be very attractive to consumers. Why didn’t Google force Verizon and Sprint to kowtow to the Nexus One before allowing them to release any more Android phones?

The real problem since then has been the wifi. When I originally installed the box, I saw that the wifi wasn’t working. I ran lspci, saw that I had a Broadcom BCM4312 chipset, and found with a quick trip to Google that it needed some proprietary firmware (joy). There were some old articles on the net about how to do that by cutting it out of other drivers, but it seemed the modern way to do this is just to enable the rpmfusion repos and install kmod-wl. So I did that, and it worked. Problem solved and largely forgotten, for a while.

However, as far as I can recall, this never worked on Fedora 14. So I was stuck with either no wifi (Fedora 14), or no presentations (Fedora 13). I chose to run Fedora 13 for my personal hacking, and use the Windows boot (which naturally worked fine) for presentations. Then a month or two ago I updated Fedora 13, and wifi stopped working there too. Even if I booted an earlier kernel; under no version could I get it working. Windows of course still worked, so I knew it wasn’t hardware.

Fedora 15 just went GA a few days ago. I installed it yesterday, tried the same trick with kmod-wl, and glory be! Wifi worked! Mirrored video worked! I was back in business.

Then I found that my Gnome 3 desktop (which by the way… not sure I like… weird merger of Mac and Android) had gotten into a weird state, and ran a system update for bug fixes. Calamity! Wifi was back to broken – again, with either the original kernel or the updated one. NetworkManager failed to even run – it would start with a red mark in my toolbar, then just disappear. Maybe it’s not kernel-specific? Maybe it’s NetworkManager? I don’t know.

Well, I just installed again, and ran updates, and then installed kmod-wl (which looks like it might have pulled in an updated broadcom-wl for the new kernel), and it looks like the wifi is back up. But this is a lousy situation – I’m going to be fearing making system updates because my wifi will break I’m not even sure who to complain to… the driver is 3rd-party, in fact so is the repo, so it’s not Fedora’s fault per se. On the other hand maybe the reason it keeps breaking is because Fedora moves too quickly and changes kernel APIs too much for the wifi firmware to keep up.

For now, I guess the solution is to dual-boot F15 with itself, and only update one at a time to make sure one is always working. (Or try Ubuntu, some of you will say; but I like Fedora, and they don’t share bootloaders very well.)

If I remember correctly – is Honeycomb the first version of Android where we actually saw a preview, got to fiddle with the SDK platform preview before it was actually embodied in a device? If so, better late than never, and let’s hope it means we’re on the way to seeing more of a community effort. Hey, it took a while for Red Hat to learn with Fedora, too, and they didn’t have voracious proprietary partners to contend with.

I have a meetup or two to arrange, but I hope I get some time to work further with ORMlite shortly.

Happy April Fools Day tomorrow!

When I tried out the demo project (which I also used in my TriDroid presentation), it looked reasonably straightforward. So just now I downloaded the JAR to take a look at what it would look like to convert LogMyLife to use OrmLite. What I didn’t notice is that the demo project is based on version 4.6, while the current version has jumped up to 4.10 a few days ago, which is apparently a significant release. I figured out things were different pretty quickly, though, because the first time I tried to annotate a DTO, the @DatabaseTable annotation couldn’t be found. I needed to download and link the ormlite-core JAR in addition to the ormlite-android JAR (which would be all that’s needed in previous releases). I suppose if I were using a maven POM to manage my build this would all be taken care of… but I haven’t gotten around to that.

I hope the docs are still reasonably accurate… I might be a little bit of a guinea pig for this release. We’ll see.

Bringing in the OrmLite helper

So I just went down a long rabbit trail about where exactly to put the OrmLite plumbing. The examples all have a helper that you have to keep track of per-Activity. But didn’t I just demonstrate for myself that an application-wide DB handle worked fine? So I’m thinking about just putting the plumbing in my Application subclass and seeing what happens. The complication seems to be multi-threading – the Google Group had a discussion pointing to this really interesting article on concurrent Sqlite access. I don’t really have multiple threads accessing the DB in LogMyLife, with the possible exception of my broadcast handlers – I’m not actually sure if they come in as a separate thread, or even a separate process, or what. That’s something for me to figure out soon. In the meantime, I guess I’ll go ahead with my scheme to try it in my Application class.

So first, I have to define a helper with callbacks for when I’m opening a connection and need to make sure the schema is in place, which needs to subclass OrmLiteSqliteOpenHelper (which subclasses the SqliteOpenHelper I’m used to) and define what to do in when the DB is being created or updated. Looks like it can also define some simple interactions for getting DAOs. This needs to be configured into the OpenHelperManager which is used to actually create my helper in context; the method of configuration is either to configure a string in my strings.xml and let magic happen behind the scenes, or to explicitly set a factory object on OHM that will return my helper. As it’s not much trouble and (I think) clearer, I do the latter.

Nuts and bolts

Soon I run into a little issue. OrmLite refers to SQLException all over the place, but it’s referring to java.sql.SQLException, generally used for JDBC, as opposed to the android.database.SQLException that’s usually used for Sqlite exceptions. It would be a nuisance just to have to be conscious of which is relevant. But the real difference is that the android.database.SQLException is a runtime exception, while java.sql.SQLException is a checked exception, so I have to deal with it somehow in any of my code that uses the DB. This forces me to re-think exception handling. Checked vs. un-checked exceptions is kind of an old argument, but I thought after all our experiences with Java the scales were starting to tip toward unchecked? As far as I know Java is the only major language using checked exceptions widely; the dynamic languages gaining popularity don’t really even have the ability to use them. So it’s kind of disheartening to see this throwback; although I imagine it arises from the JDBC nature of the rest of the OrmLite package.

Now I go to create a Dao by subclassing BaseDaoImpl, and I’m given a choice of four different constructors (but must include one). The most obvious-seeming one is evidently to be used with a DI container, so I look at the other ones. Eventually I realize that it’s probably best to use the one that’s like the generic DAO constructors, e.g. BaseDaoImpl.createDao(getConnectionSource(), Tracker.class).

While I’m working on the numerous errors Eclipse is reporting from my brutal refactoring, I realize that my BroadcastReceivers don’t have access to my Application subclass, so they have their own code for instantiating the DB connection. There’s no helper base class for them – reasonable, given they don’t really have a life-cycle. Suddenly just defining a string looks like a much nicer path for configuring the helper class.

An example query

And now I come up against one of my first query refactorings. I have a method like so in my existing domain code:

public Alert fetchNextAlert() {
 return fetchAlert(
  db_ALERT_ENABLED + " = 1 AND " + db_ALERT_SKIP_NEXT + " != 1 AND " +
  db_ALERT_NEXT_TIME + " > datetime('now', 'localtime')",
  null, null, null, db_ALERT_NEXT_TIME + " ASC", "1"
 );
}

fetchAlert takes these raw SQL constraints and turns them into a query for a single row from the Alerts table, then converts that row into an Alert DTO. It’s all fairly ugly code, to be sure. Here’s my first crack at a replacement in my OrmLite DAO:

public Alert fetchNextAlert() throws SQLException {
 return queryForFirst(queryBuilder()
  .orderBy(C.db_ALERT_NEXT_TIME, true)
  .limit(1)
  .where().eq(C.db_ALERT_ENABLED, true)
  .and().ne(C.db_ALERT_SKIP_NEXT, true)
  .and().gt(C.db_ALERT_NEXT_TIME, new Date())
  .prepare());   
 }

Now this is reasonably clear and concise, and in line with how other ORM libraries work. But one thing troubles me: as far as I can tell, there’s no way to get raw SQL in as part of a where clause; I couldn’t duplicate the ” > datetime(‘now’, ‘localtime’)” constraint, and in my experience, when it comes to time, you want to speak the DB’s language. But perhaps I’m wrong this time – certainly the code above captures the intent. If absolutely needed, I could run a raw query to return the ID of the Alert I want, then just use standard DAO methods to turn that into an Alert DTO; two queries instead of one, though.

I have reasonably-sized swaths of code compiling at this point; time for a break.

• Doesn’t having to extend the proprietary base class as your Application class seem a little… invasive? It’s like saying, this isn’t my application, this is an ActiveAndroid application!
• I’m sure it wasn’t intended this way, but looking at the licensing terms:

The SOFTWARE PRODUCT is licensed as follows:
(a) Installation and Use.
Michael Pardo grants you the right to install and use copies of the SOFTWARE PRODUCT on your computer running a validly licensed copy of the operating system for which the SOFTWARE PRODUCT was designed [e.g., Windows 95, Windows NT, Windows 98, Windows 2000, Windows 2003, Windows XP, Windows ME, Windows Vista].

Let’s ignore for the moment the question of what OS this intended for… but isn’t this JAR going to be built into our app? Doesn’t this technically mean we can’t distribute the app?

• This is based on reflection, right? So isn’t that gonna be kind of slow on Android?

You can, of course, modify this behavior by overriding setViewText, which lets you reformat the text or modify the view as you wish; but it doesn’t give you the DB cursor, just a String, so you can’t pull the data yourself or refer to other columns. But not to despair (or turn to the NotSoSimpleCursorAdapter)! You can modify anything you want by providing a ViewBinder to the adapter.

The main reason I’m talking about this is because I think the documentation on this is just a little vague:

An easy adapter to map columns from a cursor to TextViews or ImageViews defined in an XML file. You can specify which columns you want, which views you want to display the columns, and the XML file that defines the appearance of these views. Binding occurs in two phases. First, if a SimpleCursorAdapter.ViewBinder is available, setViewValue(android.view.View, android.database.Cursor, int) is invoked. If the returned value is true, binding has occured. If the returned value is false and the view to bind is a TextView, setViewText(TextView, String) is invoked. If the returned value is false and the view to bind is an ImageView, setViewImage(ImageView, String) is invoked. If no appropriate binding can be found, an IllegalStateException is thrown.

OK, great. This also promises that you can put any kind of data in any kind of view (not just a TextView). But I didn’t know what it meant by “if a SimpleCursorAdapter.ViewBinder is available.” Turns out it’s pretty simple:

1. Implement the SimpleCursorAdapter.ViewBinder interface (it has only one method, setViewValue, which gives you the Cursor and the view to work with – and just return false to let the adapter’s default behavior handle the binding). I did this for LogCursorAdapter in an inner class.
2. Instantiate your implementation and use setViewBinder on your SimpleCursorAdapter instance to set it up as the binder. This makes it “available” for the process described above.

This is arguably better than overriding setViewText because you wouldn’t even have to subclass the adapter to do it – or even create a class (it could be an anonymous implementation). And of course you can access all of the cursor columns in any way you please. Nice.

As far as my earlier data retrieval woes, this gave me the ability to pull data out the way I wanted. Sqlite seems to be storing plenty of precision in the NUMERIC column type; it was just a matter of it being retrieved as a String that caused truncation of precision. In this case the solution was just to pull it out as a Long or Double as appropriate and format it myself (I also learned about DecimalFormat which was very helpful).

Some interesting stuff I just learned recently:

What’s in a name?

The application name and the name you see on its home screen/launcher icon need not match (though it’s probably wisest if they do). The application has a label in the manifest, and so does each activity (potentially).

If you look up the application in the settings page, it will show the application name (probably also in any market interactions). The launcher icon will show the label on the launcher activity. Presumably this is because you could conceivably have multiple launching activities for the same app (different entry points).

In my case, I wanted to show a different label (“LogMyLife – Main”) to the user once they launch the app than the application name (“LogMyLife”), so they’re oriented once they return; so I just created a shell activity (“Launcher”) with the correct app label which simply replaces itself with the real front Activity, just so I could get the labelling right. There’s probably a better way, like changing the label once the activity launches. In fact, I just tried that out (setTitle at the beginning of onCreate) and it works fine. Heh – live and learn.

Also interestingly, when I noticed this and changed the label, Android didn’t handle the change very well, at least in the 1.5 emulator where I test this (likely fixed since). The icon with the old label remained, but it wouldn’t actually load the app (error), and there was no new icon/label. I had to un-install and re-install the app to get it to launch properly.

Displaying DB data

I was going to follow up a bit on my last post, but I think I’ll do it with another post.

Onward

Anyway, the release is out there, so please let me know what you think of it. And feel free to hack it up if you want to see your own improvements! Github can have your own clone set up in seconds.

I may start working on the website for this sooner rather than later. SpringSource released some kind of library to assist with Android so I’m itching to try it out!