Rails: Engine to stream live logs

Published at 04-06-2014

A company asked me to demonstrate my skills for Rails, BackboneJS, MarionetteJS and data live stream. I decided to get a try with javascript EventSource and ActionController::Live. You can take a look at the sources here. It provides an Interface to watch and filter app logs, live!

I share in this post various things I learned around this experimentation.

Browser to server with EventSource

EventSource is a new HTML5 feature that provides one-way (server to client) event streaming.
It's really simple technology, easy to implement and use. Basically you open a connection from the browser with the EventSource object and the server send JSON message through this continuous connection.

Here an exemple with my start_streaming function (see logs.coffee view):

start_streaming: ()->
    #skip if source already open
    return false if this.source

    this.source = new EventSource RL.relPath("logs/#{this.model.get('name')}")

    me = this
    this.source.addEventListener 'error', (e)->
      if this.readyState == EventSource.CLOSED
        me.logError 'Connection lost...'
        me.logError 'Connection error... Auto reconnect'

    this.source.addEventListener 'logs.all', (e) =>
      me.logError 'Reconnected' if this.error
      this.error = false
      markdown = $.parseJSON(e.data)
      matching = 'match' if !!this.filter && markdown.indexOf(this.filter) != -1
      me.logHtml "<pre class='prettyprint'><code class='#{matching}'>#{markdown}</code></pre>"

More details about this technology here (API) and there (tutorial).

Browser compatibility

Like always, IE still do not implement it. Any way, you have consider that only most recent version of Firefox and Chrome implement it. During this experimentation I found this article that propose two solutions that provides fallbacks in case of missing EventSource:

I haven't tried any of them yet, but soon ... may be ...

Rails ActionController::Live

EventSource is half of the way. Server side should handle the connection.
Here come to the rescue ActionController::Live. It's a module you can drop in any controller to turn it into a potential streaming endpoint.

Here a basic exemple with message formatting for an EventSource listener:

class StreamingController < ActionController::Base
    include ActionController::Live

    def stream
        logger.info "Stream start"
        response.headers['Content-Type'] = 'text/event-stream'

        0..100.each do |i|
           send_event "Hello for the #{i} time"
        logger.info "Stream end"

    def send_event message
         unless message.blank?
              response.stream.write "event: new_message\n"
              response.stream.write "data: #{message.to_json}\n\n"

You can take a look at this Rails casts (pro) or at my LogsController if you want to see how I connect it with the logs tail.

Read continuously a file in Ruby

Now we have a continuous connection from the Browser to the server, let's connect the logs files.
We need to read the log file and when reaching the end, wait for more!

It's something that I made in C, back to university, but I never did it with Ruby. It tried couple of technics but the easiest way came out through file-tail gem.

Some basic usage:

  File.open(filename) do |log|
    log.interval = 10
    log.tail { |line| puts line }

Backbone HAML templates

Now we get through the core technical feature, it's time for classic UI.

I love HAML and I don't want to use anything else even for backbone. I found another gem to fix this haml coffee assets. It allow to write HAML templates containing coffee script instead of ruby.

    %h2 Cart
    - if @cart.length is 0
      %p.empty Your cart is empty
    - else
        - for item in @cart
              = item.name
              %a{ :href => "/cart/item/remove/#{ item.id }" } Remove Item

HAML is not the fastest templating solution in term of performances (may be the opposite). But HAML readability and speed to write is everything to me.

Templates files are stored into app/assets/templates or app/assets/templates/live_logs in my case for nested reason with .hamlc extension. I guess precompiling assets will turn them into html files with javascript, resolving performance issue.

Backbone JST issue with nested path

Because I used a rails engine, assets are nested under the engine path live_logs. JST loading system is a bit weak and get lost with it. Here some Coffee Script to fix this.

  # Let's define a JST fixer that:
  # * Base on current path
  # * Remove root '/'
  # * ensure that both '/a_path' and '/a_path/' works

  RL.JST = (path)->
    id = window.location.pathname
    id = id.substring(1) if id[0] == '/'
    id = id + '/' unless id[id.length - 1] == '/'
    id = id + path

And here we go, let's replace default JST call by our app function

  class RL.Tab extends Marionette.ItemView
    #template: JST['tab']
    template: RL.JST 'tab'

jQuery matcher :contains

The last feature I implemented was a log filter. I discovered a jQuery feature that I didn't know: the contains selector. With those simple lines, I apply a match class to highlight the content:

  this.ui.container.find("code").removeClass 'match'
  this.ui.container.find("code:contains(#{textToFind})").addClass 'match'

And done

Really fast explained tiny things today. If you want more details, fork a version of my repository. I think I will get through tiny projects like this one more often, really refreshing. If you find something useful or have question on this short article or Live logs Rails engine, send me your feedback.