I have been recently working on the Credit Union Findr application and I think it is a pretty interesting problem to solve. To give you some idea about the functional requirements, the application intends to help people find credit unions they are “eligible” for. If you don’t know what a credit union is, it is a non-profit financial institution that is owned and operated entirely by its members. They provide a variety of financial services, including lending and saving money at a much better rate than regular banks. Credit unions can only accept members that satisfy certain eligibility criteria. The eligibility criteria are based on where you live, work, worship or volunteer. They are also based on who you work for, your religious, professional affiliations, et al.

So on the face of it, it seems like a simple enough problem. Take the eligibility criteria for credit unions, take the user’s personal details and then match the two. Sounds simple, right? Not so much. When we started building the application, there were some interesting challenges on the domain side (non-software) as well as on the implementation side (the software bits). Although the focus of this blog is to talk about the software side of the application, I will quickly run through the domain challenges.

First of all, gathering the credit union eligibility data is a monumental task. Thankfully, the product owner team, Alternate Banking System (ABS) handled that for us. Second, we were not sure which questions to ask the user. Because we could ask them everything under the sun and still not have any qualifying credit unions as a result, which is rare, but still a possibility. Moreover there were concerns around how much personal information would the user be comfortable sharing with us. All these challenges were tackled by our UX (user experience) team when building the prototype and they made the suggestion that we should treat this application like a dating site. You ask the user a minimum set of relevant personal questions, ask them what they were looking for in a credit union and then match the two. The analogy made it much simpler to visualize the system.

The match making analogy stuck with us developers. But the big question from an implementation stand point was how do we model the data. Max Lincoln, one of the brilliant engineers on our team, made a suggestion that we use a graph database. Frankly at first I found that a bit odd. I was thinking more along the lines of a document store. You throw credit union eligibility criteria on a document, user details on another document and match the two. Graph database. Really? But any way what do we do when we have two approaches to solve a problem. Well, we spike them out! Both the spikes were quick and revealed the merits and demerits of each solution. Mustafa Sezgin led the document store spike and used ElasticSearch. As I thought more and more about the problem, I became convinced that graph solution was the way to go. Let me explain how.

So lets start off by an example to see how we could model this problem as a graph. Lets say, you have a credit union that accepts people working in Manhattan. And you have a user who works in Manhattan. Now on a graph, you would have a node for the credit union which has a relationship called “acceptsWorkingIn”, represented by an edge, to the node Manhattan. And on the user side, you would have a user node, that has a relationship called “worksIn” to the same Manhattan node that also connects to the credit union. Now lets say you had another credit union that accepted people working in Manhattan. Then you would have a new credit union node to represent it and with a relationship “acceptsWorkingIn” to the same Manhattan node. Now to find the credit unions that the user is eligible for, you would simply traverse the graph starting from the user node until you find all leaf credit union nodes.

One of the reasons why the graph solution really shines is “hotspot questions”. Let me first explain what a hotspot is. So in our example, a hotspot is a node to which you have a lot of credit union nodes attached to. So in our previous example, if we keep adding all the credit union eligibility data as far as “acceptsWorkingIn” relationship is concerned, and you find out that 90% of the credit union nodes are attached to Manhattan node, then Manhattan node clearly becomes a “hotspot”. Remember what I mentioned earlier, that we had a difficulty coming up with which questions should we be asking the user. Hotspot questions! The questions that lead us to a hotspot quickly are hotspot questions. So a hotspot question in this example would be “Do you work in Manhattan?”. The UX folks were delighted because if we could ask a hotspot question as our first question and get a positive response, the user becomes eligible for 90% of the credit unions right away. More results quickly equals big win for usability! Now there is another side to this hotspot questioning. Lets say all credit unions accept people who live on the Ellis Island, and what if nobody lives on Ellis Island, you would get a negative response for all the users. Doesn’t make it a great question, does it? Now lets say we had a healthy database of users and you analyze the data from the user side and you find out that a large percentage of users work for NYC Metropolitan government and end up qualifying for some credit union. So you can find the user an eligible credit union if they also happen to work for NYC Metropolitan government. So the hotspot question would be “Do you work for NYC Metropolitan government?”.

Other reason why I liked the graph model is because a graph model is flexible and easy to extend. Lets say you have a credit union that accepts folks who work for NYC Metropolitan government and you have a user who works for NYC Transit. Now in a graph model, you could easily have a connection between NYC Metropolitan government and NYC Transit via the relationship “belongsTo”. This way, over time as you gather more information, you could have multiple levels of relationship between the user and the credit union. The technical implementation remains exactly the same. All you have to do is traverse the graph from the user node to the leaf credit union nodes.

What I have realized is that, what would be a value in a document store, is a first-class entity in a graph model. So for example, in a document store you have an attribute “worksIn” on a user document with the value Manhattan. On a graph database the value Manhattan gets its own node which could have its own properties, could have its own relationships to any other node in the system. More so, there is one unique node to represent Manhattan, thus giving rise to the concept of hotspot which has been immensely valuable in our case.

Any way we have just started building this application and I am really excited about it. For the technology stack we are using Scala, Play! and Neo4J and will be deploying to Heroku.

Thanks for reading and comments welcome!