Yodlee and Rails Implementation, Part 1: Getting Started

Dubbed by CNN the "800-pound gorilla of the internet", Yodlee is a service like no other: it is used by banks, financial institutions, and apps to aggregate consumer data from across the financial spectrum. It gathers and collates everything from student and mortgage loans to credit cards, 401(k)s, and checking and savings accounts.

Back in December 2012, I published a blog post on using Yodlee with Rails. It was intended as the first post in a series, but due to time constraints, the rest of the series never materialized.

Since then, Yodlee has released its new REST/JSON API to replace its SOAP/XML API.

At Firehawk Creative, we just completed a Yodlee integration for a client using the new REST/JSON API, and I was thrilled to discover that the new API is indeed much, much easier to use.

What This Series Is

In this series we are going to look at how to to create a Yodlee Rails app. We will traverse the full workflow: interacting with banks, adding users and accounts, and finally, extracting transaction data from those user accounts.

The series will have a heavy focus on doing this the Ruby + Rails way. As Yodlee has their own system engineered in Java, the docs have a certain Java flavoring, but here we are going to do this the Ruby way.

Time to Rock

Alright, ready to get started? Great.

The first thing you need to do is head over to the new Yodlee developer portal and open an sandbox development account. This will give you sandbox credentials, which provide access to the sandbox API for 30 days.

The sandbox environment is identical to the live production Yodlee environment, with the exception that the sandbox comes with five demo users only. In the production environment, you will obviously be able to add as many users as your app needs.

Gems Are So Pretty

We will be making use of two gems. The first is httparty, one of the many gems available for making web requests. I like httparty because of it's simplicity and easy of use, though if you prefer another gem translating the code in this series won't be difficult at all.

The second gem we will use is hashie. Hashie is a collection of wrappers for the standard Ruby hash that makes the hash more useful. We will be making liberal use of the Hashie::Mash variety.

Gemfile
gem 'httparty'
gem 'hashie'

Configuration

The first step is to set up a config YAML file to store the Yodlee credentials you got earlier from the Yodlee developer portal. In your config folder, create a filed named yodlee.yml, like so:

config/yodlee.yml
development:
  base_url: https://rest.developer.yodlee.com/services/srest/restserver/v1.0
  username:
  password:
  register_users: false

test:
  base_url: https://rest.developer.yodlee.com/services/srest/restserver/v1.0
  username:
  password:
  register_users: false

production:
  base_url:
  username:
  password:
  register_users: true

So what are these different values we are saving?

  • base_url is the base Yodlee API URL. For the sandbox, this is https://rest.developer.yodlee.com/services/srest/restserver/v1.0. When you are ready to go live, Yodlee will supply you with the production base URL.
  • username is your Yodlee Cobrand Username, usually something like sbCobjohndoe
  • password is your Yodlee Cobrand Password, usually something like 1234abc5-1234-1ab2-a12c-e1ab23cdef4f
  • register_users when a user signs up for your app, you need to register the user with Yodlee so that you can later add bank accounts for the user. This setting defines whether we should automatically register users with Yodlee when they are added via User.create. This is set to true for production only, as when you use your sandbox credentials in development, you must use only the five users Yodlee pre-creates for you.

Connecting To Yodlee

To start, let's make one basic API call to Yodlee, simply to login.

Create a folder inside your app/models directory named yodlee. All our Yodlee code will be going here, keeping it separate from our core app code.

Create a file inside our new yodlee folder named base.rb. This will be our Yodlee::Base class that all our forthcoming Yodlee classes will inherit from.

app/models/yodlee/base.rb
module Yodlee
  class Base

    include HTTParty

  end
end

Notice that we have wrapped the Base class inside a module named Yodlee. All our Yodlee classes will be part of the Yodlee module. This is simply to namespace the Yodlee-specific code, gem style, which prevents code clashes with other classes.

Now before we can actually make any API calls, we need to read in the credential values from the yodlee.yml file we just saved.

Create another file and name it config.rb.

app/models/yodlee/config.rb
module Yodlee
  class Config

    class_attribute :username, :password, :base_url, :register_users

    # Load yaml settings
    YAML.load_file("#{Rails.root}/config/yodlee.yml")[Rails.env].each do |key, value|
      self.send("#{key}=", value)
    end

    Yodlee::Base.base_uri base_url

  end
end

This config file reads in the YAML file, grabs the values specific to the environment, and saves them on the Yodlee::Config class level. This, for example, allows us to use Yodlee::Config.username anywhere in our app to get our Yodlee username.

You'll also notice that we added Yodlee::Base.base_uri base_url. This is an HTTParty method and allows us to set the base URL for HTTParty API calls. If we didn't set this, we'd have to specify the full https://rest.developer.yodlee.com/services/srest/restserver/v1.0 on each and every API call.

Back in our Base class, let's add a login method.

app/models/yodlee/base.rb
def login_app
  credentials = {
    :cobrandLogin => Yodlee::Config.username,
    :cobrandPassword => Yodlee::Config.password
  }

  response = self.class.post('/authenticate/coblogin', query: credentials)
end

We use self.class.post to make a POST request. This is HTTParty syntax. The first parameter, /authenticate/coblogin is the Yodlee endpoint for login API calls. HTTParty automatically appends /authenticate/coblogin to the base_uri value we set in the Yodlee::Config class, so that it sends the API call to https://rest.developer.yodlee.com/services/srest/restserver/v1.0/authenticate/coblogin.

The second parameter is the parameters for the call, in this case our login username and password. Note that Yodlee expects everything to be camel-cased, so we use cobrandLogin rather than cobrand_login.

In your Rails console, you can now run the login method.

reload!
Yodlee::Base.new.login_app

If all goes well, you should get a response along the lines of:

#<HTTParty::Response:0x7ff63e233e30 parsed_response={"cobrandId"=>10010352, "channelId"=>-1, "locale"=>"en_US", "tncVersion"=>2, "applicationId"=>"3A4CAE9B71A1CCD7FF41F51006E9ED00", "cobrandConversationCredentials"=>{"sessionToken"=>"08062013_0:4ce859dfa9dd9f3a1fc48314890e31ee22c416833c9a834c5d3486cfa7c56040ba0d392bc0ff8d638c7a28cb7b14123228f6556c9cf459d1e0135e578942239e"}, "preferenceInfo"=>{"currencyCode"=>"USD", "timeZone"=>"PST", "dateFormat"=>"MM/dd/yyyy", "currencyNotationType"=>{"currencyNotationType"=>"SYMBOL"}, "numberFormat"=>{"decimalSeparator"=>".", "groupingSeparator"=>",", "groupPattern"=>"###,##0.##"}}}, @response=#<Net::HTTPOK 200 OK readbody=true>, @headers={"x-powered-by"=>["Unknown"], "set-cookie"=>["JSESSIONID=736A5C67C53203DB3D320C15F0D92694; Path=/services; Secure"], "content-type"=>["application/json"], "date"=>["Mon, 25 Nov 2013 22:28:40 GMT"], "connection"=>["close"], "server"=>["Unknown"]}> 

Refactor, Refactor, Refactor

While this code works, making our calls to Yodlee in this way will get tiring fast. Do we really want to use the self.class.post syntax everywhere? And how do we extract items from the response nicely? And what about error handling?

Gosh, so many questions :)

I'm going to do a major refactoring now to flesh out our Base class with useful methods for the API calls. So hold on tight.

First, let's add a query method. This is the only method that will actually be making HTTParty web requests. All other methods will simply defer to this one. This allows us to define our web request behavior in a single spot, and if we ever need to tweak it or update it, we can do it in just one place.

This query method will accept a few options. method, which can be set to POST or GET or another valid web request method type, endpoint which can be set to the exact endpoint hit, like the /authenticate/coblogin endpoint we used above, and finally it will accept params which are the body parameters of the API request.

app/models/yodlee/base.rb
def query opts
  method   = opts[:method].to_s.downcase
  response = self.class.send(method, opts[:endpoint], query: opts[:params])
  data     = response.parsed_response

  if response.success?
    if [ TrueClass, FalseClass, Fixnum ].include?(data.class)
      data
    else
      convert_to_mash(data)
    end
  else
    nil
  end
end

Let's go through that line by line.

First, we take the method parameter, usually specified as POST or GET, and downcase it to post or get. We can now use that on the next line, in our send call. This is the call that actually triggers HTTParty to make the API call. We then assign the response block to a variable named data.

If the response was not successful, in other words, the web request did not return a 200 success code, we simply return nil from this method. Otherwise, we inspect the class of the response. If the class is a True, False, or Integer value, we simply return it as-is. Otherwise, it's a hash of data, so we convert it to a Hashie::Mash.

app/models/yodlee/base.rb
def convert_to_mash data
  if data.is_a? Hash
    Hashie::Mash.new(data)
  elsif data.is_a? Array
    data.map { |d| Hashie::Mash.new(d) }
  end
end

Why do we want to convert our hashes to Hashie::Mashes? A Hashie::Mash allows us to use object-like syntax, which is much more readable:

# Instead of having to write something like this:
response[:userContext][:conversationCredentials][:sessionToken]

# We can write this: 
response.userContext.conversationCredentials.sessionToken

We can now rewrite our original login_app method to use our new query method.

app/models/yodlee/base.rb
def login_app
  query({
    :endpoint => '/authenticate/coblogin',
    :method => :POST,
    :params => {
      :cobrandLogin => Yodlee::Config.username,
      :cobrandPassword => Yodlee::Config.password
    }
  })
end

Isn't that much more readable? Look how clearly it now states its intent.

The Cobrand Session Token

We can now log in our app with Yodlee successfully, but what does that even mean?

We log in the app so that we can get back a Cobrand Session Token. This token represents your app, and needs to be passed along with every future API call.

The session token lasts for 90 minutes, so we want to store it for that long and use it across all API requests being made by all of the app's users for the entire 90 minutes to reduce API calls. (Yodlee recommends to call this API only once in every 90 minutes.)

This is an ideal use-case for a class variable. In general, class variables in Ruby get a really, really bad rap and are pretty much considered the work of the devil because subclasses can overwrite a superclass's class variables.

In our case however, that is exactly what we want. We want all our subclasses to have access to the token and we also want all our subclasses (and their instances) to be able to overwrite the token when it expires and set it to a new token.

So this is one of the really rare cases when class variables are perfect.

app/models/yodlee/base.rb
module Yodlee
  class Base

    cattr_accessor :current_session_token, :current_session_started

    def fresh_token?
      current_session_token && current_session_started && current_session_started >= 90.minutes.ago
    end

    def login_app
      response = query({
        :endpoint => '/authenticate/coblogin',
        :method => :POST,
        :params => {
          :cobrandLogin => Yodlee::Config.username,
          :cobrandPassword => Yodlee::Config.password
        }
      })

      self.current_session_started = Time.zone.now
      self.current_session_token = response.cobrandConversationCredentials.sessionToken
    end

  end
end

Using ActiveSupport's cattr_accessor method we set attribute accessors on the class level. These are backed by class variables.

We updated the login_app method to extract the session token from the Yodlee response and assign it to the current_session_token class variable. We also assign the current time to the current_session_started class variable so the fresh_token? method can calculate whether or not the token is still fresh.

Finally, we add a simple method to wrap this all together:

app/models/yodlee/base.rb
def cobrand_token
  fresh_token? ? current_session_token : login_app
end

Remember that all our future Yodlee classes inherit from this Base class, and all those classes will need to send in the cobrand session token with every API call. They can now do this by using cobrand_token, which will return the current token if it's fresh, or fetch and return a new token if it's stale. Sweeeet.

Tip: In the code above, we did error handling on a very general level by checking if a 200 response code was returned by Yodlee or not. Keep in mind that most times when an error happens, Yodlee will return a hash with the error messages and a 200 success code. Dealing with these exceptions is beyond the scope of this series, but you do need to write code to effectively handle those edge cases.

Wrap Up

At this point, our Base class looks like this.

app/models/yodlee/base.rb
module Yodlee
  class Base

    include HTTParty

    cattr_accessor :current_session_token, :current_session_started

    def cobrand_token
      fresh_token? ? current_session_token : login_app
    end

    def query opts
      method   = opts[:method].to_s.downcase
      response = self.class.send(method, opts[:endpoint], query: opts[:params])
      data     = response.parsed_response

      if response.success?
        if [ TrueClass, FalseClass, Fixnum ].include?(data.class)
          data
        else
          convert_to_mash(data)
        end
      else
        nil
      end
    end


    private

    def convert_to_mash data
      if data.is_a? Hash
        Hashie::Mash.new(data)
      elsif data.is_a? Array
        data.map { |d| Hashie::Mash.new(d) }
      end
    end

    def login_app
      response = query({
        :endpoint => '/authenticate/coblogin',
        :method => :POST,
        :params => {
          :cobrandLogin => Yodlee::Config.username,
          :cobrandPassword => Yodlee::Config.password
        }
      })

      self.current_session_started = Time.zone.now
      self.current_session_token = response.cobrandConversationCredentials.sessionToken
    end

    def fresh_token?
      current_session_token && current_session_started && current_session_started >= 90.minutes.ago
    end

  end
end

Now try running this in your console.

# This should return a very long token
reload!
Yodlee::Base.new.cobrand_token

# Run the command again, and notice how it returns the cached token immediately, as we have cached it for 90 minutes, and does not ping Yodlee for a new token.
Yodlee::Base.new.cobrand_token

Coming Up Next

In our next blog post, we will start interacting with banks and other services. We will also discuss some very useful logging techniques.