Learning functional programming

Advanced track

In this series I want to explore fundamental concepts and tools of functional programming by building small applications.

Streams

A stream is a conceptual device that abstracts the way in wich we access and process data. A stream at its core is just a sequence of data. However, differently from an array or a list, a stream doesn't necessarily have a beginning or an end.

In real life we have some example of streams. A ticker for the price of Bitcoin for example. It's a stream because it's clearly a timed sequence of data. It does have a beginning (when Bitcoin was created) but it never ends.

Building a Bitcoin price ticker

In this issue you'll build a BTC ticker that runs on your screen. This way you will always know when to buy and when to sell.

Design

A simple design you can draft in two seconds is the one sketched above. We'll chose some API, parse the output, turn it into a number and finally print it on our screen (or physical ticker if you are retro).

Programming

A very first draft of the program might be like this:

ZStream
  .repeat("100.000") // generate some fake data
  .take(3) // let's not overload
  .map(identity) // placeholder for parsing
  .map(priceString => BigDecimal(priceString))
  .foreach(price =>
    putStrLn(
      s"BTC-EUR: ${price.setScale(2, BigDecimal.RoundingMode.UP)}"
    )
  )

You can go ahead and launch this (opens new window). In the first line we are getting a hold of the ZStream object, it contains the repeat method wich just infinitely produces whatever we pass: a string in this case. We limit it to 3 strings, otherwise it will just overload our machine and never stop. The parsing for now is impersonated by an identity function. We then leverage the BigDecimal parsing capabilities to have a number to work with. Lastly the foreach method is used, which is a terminal method. It will wrap up the stream by printing on the screen a well formatted price tick.

We could have even skipped the conversion to BigDecimal and back if we don't care about having control on rounding.

ZStream
  .repeat("100.000")
  .take(3)
  .map(identity)
  .foreach(price => putStrLn(s"BTC-EUR: $price"))

Now we go hunting for a good API. Like in the cooking TV show I’ve already done this, but you can look for yourself and I recommend ProgrammableWeb (opens new window).

I’m going to use the Kraken public API (opens new window) which is very basic. It returns a json of this form:

{
    "error": [],
    "result": {
        "XXBTZEUR": {
            ... other stuff
            "c": "last trade price",
            ... other stuff
        }
    }

We are interested in the current price which is the last trade price. We could (and should) use a json parser to extract this. However that would open a can of worms, so I’m going to use a regular expression:

def parsePrice(json: String): String = {
  val regex = raw"""(?s).*"c":\["([0-9.]+).*""".r
  json match {
    case regex(price) => price
  }
}

So we have our parsing step, we finally need to actually call the API. Again, there are super-cool http clients we could use; instead for simplicity's sake we'll use the scala built-in client.

val tickerURLCall: Task[String] = IO(
  io.Source
    .fromURL("https://api.kraken.com/0/public/Ticker?pair=BTCEUR")
    .mkString
)

We can now put everything together in the program. We will use tickerURLCall instead of the fake data and the priceParce instead of the identity.

  val program: ZIO[Console with Clock, Throwable, Unit] = ZStream
    .repeatEffect(tickerURLCall)
    .throttleShape(1, 5.seconds)(_ => 1) // let's slow down
    .map(parsePrice)
    .foreach(price =>
      putStrLn(
        s"BTC-EUR: $price"
      )
    )

Node the addition of the throttleShape function. This is because we don't want to call the API as fast as we can, but we want a constant slow rate. The code says 1 call every 5 seconds. The function we supply in the second pair of brakets is a function to compute the weight of the result (to shape the throttle accordingly) we conside all responses equal to 1.

This last feature is useful if you have pagination and you don't know how many items you have for each page. You could create a stream that outputs 10 items per minute and sometimes the calls to the source API return more (or less). Then the API would be called more slowly (or faster) to compensate.

Also we don't need to limit the number of ticks with take(3) now that we have throttling in place.

You can test the bitcoin ticker on Scastie (opens new window) remember to edit it to display your favourite currency pair and your favourite rounding.