Mitchell Hashimoto

"New" Purely Functional Data Structures

Sat, 13 Aug 2011 10:38:28 -0700

"New" Purely Functional Data Structures:

This StackOverflow post covers “new” purely functional data structures created since Okasaki’s book “Purely Functional Data Structures” published in 1998.

"The Essence of Functional Programming" Part 1: The Identity Monad

Tue, 12 Jul 2011 23:06:00 -0700

This is the first of a new series of posts that will take papers that I read and will explain them and attempt to make them more concrete (and less theoretical) by showing real world examples using more industry-standard languages. These papers will be tagged with theory-in-practice. For more information on the “Theory in Practice” series, read the introductory post.

Today’s paper is “The essence of functional programming” by Philip Wadler. This paper is widely known as the paper which popularized monads, and introduces them in a relatively elementary context with a focus on practical programming use.

This is part 1 of a multi-part series. In order to allow the information to soak in and to not consume too much of your time, I’ve decided to split this into multiple parts which I will publish over the coming weeks. These will all be tagged with essence-of-functional-programming.

Part 1 will cover the inspiration for monads as well as introduce the trivial monad.

Prior knowledge required: Basic programming experience with basic experience in functional programming (in any language, whether it be Ruby, Python, C, etc.). The examples will be given in Haskell and Scala.

Inspiration: Purity vs. Impurity

The inspiration for monads comes from the desire to program in a purely functional environment, but to ease program modification in cases where it appears impure languages have the upper hand.

What is “purity?” What is “impurity?”

While a conceptual “pure vs. impure” discussion is out of the scope of this post and the definition of purity was not covered in the paper, I’ll quickly define the terms. A function is pure if it generates it’s result using nothing but the parameters given to it, whereas a function is impure if it uses or modifies some outside state. A quick example of a pure function:

def add(x: Int, y: Int) = x + y

And that same function becoming impure, since we’re modifying global state by outputting text to the world:

def add(x: Int, y: Int) = {
  println("Adding!")
  x + y
}

Additionally, the benefits of purity are:

Increased modularity - A pure function can be used anywhere and it will be have the same way. Therefore pure functions are ideal for libraries among programs.
Concurrency safe - Since a pure function modifies no outside state, it can be easily (theoretically) parallelized.
Simplified testing - Once you test all input conditions and edge cases, you can be certain the function behaves correctly.

Of course, there are also benefits to impurity:

State - Its very easy to maintain state in impure situations, since you just add a global variable to the mix, and every function can use it.
Quick edits - You can just throw in a println wherever you see fit to assist in debugging.

A pure pain point

As an example for comparing purity vs impurity, let’s use a basic interpreter which takes as input a program, evaluates it, and returns the result. This is surely a simple venture in both pure and impure settings.

Now, what if we wanted to add error handling to this interpreter? With impurity, we could simply throw exceptions with the error message. But in a pure environment, we would have to modify the result type to also include errors at each recursive point, and each function would have to check for errors prior to running. Holy smokes batman, that’s annoying.

Okay, what about adding an execution count? With impurity, all that needs to be done is to add a global variable that is incremented with each operation. Again, though, for a pure language, we would have to modify the result type to include execution count everywhere.

Based on these examples, it looks like impurity wins completely! But, there is another option: Monads. Wadler argues that monads can be used to structure an interpreter so that the above changes can be made just as easily as with an impure language, while maintaining the purity of the computations.

The Monad

A monad consists of three parts. In Haskell, it is defined like so:

class Monad m where
  unitM :: a -> m a
  bindM :: m a -> (a -> m b) -> m b

The rough equivalent in Scala would be the following:

trait Monad[M[_]] {
  def unitM[A](a: A): M[A]
  def bindM[A, B](a: M[A], f: A => M[B]): M[B]
}

As you can see:

M is a type constructor. That is, its not a complete type on its own, because it requires another type to complete it. An example, List in Scala is not a type, but List[Int] is. Therefore, List is a kind of type constructor.
unitM is a function which takes a value and “lifts” it into the monad. So if you had a just an int, such as 7, and your monad was a list of ints, then unitM would make it a list [7]. In fact, although we won’t talk about it in this post, List is a monad!
bindM is a function which allows computations to be done on the value held by the monad. Building on the list example, if you wanted to double each item in the list, you could do bindM([7], (x: Int => x + x)) and this would return [14]. Notice the computation was done on the items inside, but the result was still contained in the monad type, which was List for our short examples here.

So, unitM can be thought of as bringing a value into a monad and bindM can be thought of taking that value, doing some computation, and putting it back into a monad. The question remains: how do I get a value out of a monad? This operation is typically specific to each monad, so it is not generalized. You’ll see examples later.

Okay, I realize this was really abstract. The main takeaway from this section is that a Monad is nothing more than the three parts above. Don’t worry about understanding the “why?” of monads yet, concrete examples are on the way!

A basic interpreter

Our concrete example is going to be a basic interpreter. The interpreter was originally written in Haskell in Wadler’s paper. I’ve converted this to Scala. Both sources are available in this gist. Please read the interpreter over until you understand what it should do. You don’t need to understand the monadic parts, of course, but you should be able to get the gist of what it does. As an example, take the following input into the interpreter:

term = App(Lam("x", Add(Var("x"), Var("x"))), 
           Add(Con(10), Con(11)))

The above is roughly equivalent to the following Scala code:

term = ((x: Int) => x + x)(10 + 11)

With that input, running the interpreter should yield 42, since it adds 10 and 11, to make 21, and that is passed as the parameter into the lambda function, which doubles it to 42.

We will take the above interpreter, extend it in various ways using monads, and compare this approach to what would be necessary in an impure language.

Variation zero: Standard Interpreter with the Identity Monad

To begin, let’s create the trivial monad. This monad doesn’t actually do anything, it just wraps the value, but its a gentle introduction to monads and their operations:

case class Id[+A](val value: A)

object IdOps extends Monad[Id] {
  def unitM[A](a: A) = Id(a)
  def bindM[A,B](a: Id[A], f: A => Id[B]) = f(a.value)
  def showM[A](a: Id[A]) = a.value.toString
}

This monad is called the “identity monad.” It is called this because it doesn’t serve any real purpose except to wrap a value. While it may not be necessary, I will explain each operation for the purpose of being explicit about each monad operation:

The type constructor M in this case is Id. As you can see, this matches the above definition of a monad by taking a single value of some type.
unitM takes a value and wraps it in Id. This brings the value into the monad, just as was described earlier.
bindM allows computations to be done on the value of the monad. In this case, the function is applied directly to the value. Since this is the identity monad, there is nothing sneaky going on here, just simple function application. If we had an Id(7) and did bindM(Id(7), (x: Int => unitM(x * 2))) then the result would be Id(14). Note the symmetry between this and the example given earlier when defining monads.

For the purpose of our interpreter, we’ve also introduced a showM function. This is not a standard monad function in general, but we’re requiring all monads implement this so that we have a way to get the value out of the monad as a string.

Despite its practical uselessness, the identity monad serves a practical purpose for this post since it demonstrates the simplest monad. The result of using this monad with our example is that it constructs a basic, working, interpreter.

Testing the Monad

In order to test the monads we’ll be using during this series, I’ve constructed the interpreter example in such a way that you simply have to subclass the Interpreter trait and point it to our latest monad to see the changes. Add the identity monad and create the interpreter:

class StandardInterp extends Interpreter {
  type M[A] = Id[A]
  override val monadOps = IdOps
}

Then create a basic test program and run it:

val current = new StandardInterp()
import current._

val term: Term = App(Lam("x", Add(Var("x"), Var("x"))),
                     Add(Con(10), Con(11)))

println(current.test(term))

The result should be “42.”

This is what we expected, since our monad does nothing in this case except introduces us to the basic operations and usage of it. It should be clear to see that in the future we’ll add considerably more logic to the monad, and the interpreter will quite simply be augmented with additional functionality without at all compromising the purity of it’s functions.

Look for part 2 in the coming days!

Theory in Practice

Thu, 30 Jun 2011 06:23:23 -0700

I’m starting a series of blog posts, published weekly, called “Theory in Practice.” These will take papers I read from the world of academia and attempt to reiterate them in simpler, more practical terms. One way I will do this is to take the examples, often in Haskell, and rewrite them in a more practical language, such as Scala. I’ll also add more real world applications, if needed.

Since my interests are currently in the world of programming languages and functional programming, I will start this series with a set of foundational papers in this subject area.

My goal with this series is to open the world of academic papers to a more general audience, as I’ve found that reading these papers has helped me in my day-to-day work greatly, and I hope that my blog posts will have a similar effect on others.

The first part the series will be on “The essence of functional programming” by Philip Wadler, and will be published this week.

Welcome to Ruby. We Have Many Rubies.

Sun, 13 Mar 2011 22:13:00 -0700

Although I consider myself (and I’m proud to be) a polyglot, most of my open source code is Ruby. I’ve been a Rubyist for almost 5 years now and it is still my language of choice for starting projects. Professionally, I’ve begun to use Python in the work-place, and in the process of learning Python I’ve had a hard time understanding the various Python implementations. Additionally, I know friends coming to Ruby who have trouble understanding the reasoning behind all the different Ruby implementations.

As such, I’m going to give a brief overview of the major Rubies available, their purpose, their development status, and my opinion of them. I’ll cover Matz’s Ruby (MRI or CRuby), Ruby Enterprise Edition (REE), JRuby, Rubinius (rbx), MacRuby, IronRuby, and MagLev.

The world of Rubies is unlike other languages, such as Python, where I would guess 95%+ use CPython exclusively. It is not uncommon for a single Ruby developer to use Rubinius at home, JRuby for production deployment, and MacRuby for desktop development.

Matz’s Ruby Interpreter

Matz’s Ruby interpreter, or more commonly called MRI, is the original Ruby interpreter. This is the one you get by default from the official Ruby website or the one you probably get if you install Ruby via your package manager.

MRI is also where all the new Ruby features land first, and as such is currently the place where Ruby moves forward. Other implementations strive to reach compatibility with all Ruby that works with MRI.

Technical detail: You may see some people reference “YARV.” YARV is the Ruby VM powering Ruby 1.9, and stands for “Yet Another Ruby VM.” YARV technically replaced MRI (which powered 1.8), although Ruby 1.9 is still known as MRI.

My opinion: MRI is still the most mainstream Ruby. It is maintained by the Ruby-dev team and when someone is a “Ruby programmer” they’re usually referring to this Ruby. I personally still run this Ruby as my default at home.

Ruby Enterprise Edition

Ruby Enterprise Edition (or REE for short) is a fork of MRI (Ruby 1.8) which adds some optimizations which make running Ruby in production more performant. Some of the optimizations it adds:

Copy-on-write friendly garbage collector, which makes forking much less expensive.
Improved memory allocator
Tunable GC settings, important for servers!

My opinion: I don’t have a lot of experience with REE. Many people, such as Twitter, use it in production with great success. I’ve heard nothing but good things about it, but prefer JRuby for deployment myself.

JRuby

JRuby is a complete Ruby implementation written in Java on top of the JVM, which provides complete interaction with Java libraries.

For those new to the concept of additional languages on top of the JVM, this may seem like a huge “why?” The JVM is mature, battle-tested, and exceptionally performant. The Java ecosystem is filled with every library imaginable and is the de facto enterprise language. By putting Ruby on top of the JVM you gain numerous benefits:

Portability and speed of the JVM
Every Java library becomes a Ruby library

My opinion: JRuby is amazing. It can even compile most MRI C extensions (think about that for a moment). Also, I’ve found deploying web applications as war files the easiest method for deployment available. JRuby also beats MRI in many benchmarks. Its fast!

Rubinius

Rubinius is a new virtual machine build from the ground up for Ruby. It is written in C++ and can JIT Ruby code to LLVM bytecode. Rubinius can execute Ruby fast, far faster than MRI. The performance hits to Rubinius come from C extensions, which are slower than MRI.

Brian Ford, one of the developers behind Rubinius, posted a good Why Use Rubinius? article that is worth reading.

My opinion: Rubinius is clearly the future of Ruby VMs. The C++ implementation is beautiful and extensible, whereas the MRI code base is a mess of spaghetti C. Rubinius is extremely performant, and there are a lot of forward-thinking ideas within the VM itself. Rubinius runs most Ruby code, but still hasn’t replaced MRI for me as my default Ruby for edge-case reasons. I recommend giving Rubinius a shot if you havent.

MacRuby

MacRuby is a fork of Ruby 1.9 made by Apple, Inc. MacRuby compiles to LLVM bytecode and can interface with all of Cocoa. MacRuby can even do things like Grand Central Dispatch elegantly.

The point of MacRuby is to allow building Mac applications with a Ruby that is deeply integrated with Mac, such that performance doesn’t suffer and resulting applications are truly native.

Cool use case: Twitter uses MacRuby to unit test much of their Cocoa libraries that their iOS and Mac native clients use. This makes sense thanks to Ruby’s beautiful test libraries.

My opinion: If you’re making a Mac app, you should really use MacRuby. You can punt the super high-performance stuff to Objective-C and have MacRuby interface with it. Its a no-brainer if you’re a Ruby developer looking to make a Mac app.

IronRuby

This is here for completions sake. IronRuby is Ruby on top of the .NET runtime. This is exactly analogous to JRuby and the JVM.

MagLev

MagLev is built by GemStone and it is Ruby on top of a smalltalk virtual machine. It has a JIT as well as an impressive built-in persistence system across multiple VMs.

My opinion: I’m not aware of any people using MagLev in production personally, but its certainly worth watching.

Playing with Rubies

Want to play with other Rubies easily? Just use the Ruby Version Manager (or rvm for short) which makes it dead simple to install multiple Ruby versions and implementations side by side.

Now, can someone make one of these posts for Python? Thanks!

CloudFormation: The Big Picture

Sat, 26 Feb 2011 10:09:00 -0800

Amazon announced CloudFormation to the public yesterday, and while the general opinion I could glean from various sources shows that people are excited about this new technology, many are still unsure what it is and how it fits into their current cloud workflow. I feel as though I have a firm grasp on CloudFormation and will attempt to answer some questions here.

Note: I’m definitely not a representative of Amazon in any way, and anything here is simply my educated opinion on the matter.

What is it?

CloudFormation is best described as an infrastructure provisioning tool based on a declarative language (expressed in JSON, in this case). CloudFormation takes the declarative file (a “template”) and uses it to orchestrate the creation of various cloud services and hook them up together (resulting in a “stack”).

The difference between this and Chef or Puppet, is that Chef and Puppet are configuration management tools and work best with installing and configuring software on new nodes. Some people have used configuration management for launching virtual machines and servers and so on, but this is an exception, and not the norm.

Chef and Puppet are complementary tools to CloudFormation. CloudFormation starts the server orchestration act and configuration management tools complete it.

Where does it fit in my current workflow?

It doesn’t, yet. CloudFormation is really great at describing the infrastructure of a single application, service, or cluster, and bringing up the entire infrastructure as one atomic operation. However, CloudFormation doesn’t yet provide any way to incrementally improve and re-provision infrastructure, which is most of the battle when doing any Ops work. There are some hints from Amazon that they are thinking about this for the future.

However, if you’re looking to bring up a new service or application, CloudFormation is far superior to writing manual launch scripts. This is very nice.

So, does CloudFormation matter at all?

Yes! Infrastructure provisioning is a space which is currently lacking a mainstream tool. CloudFormation is a good step in that direction, and therefore everyone should learn about it, but it is not the end-all-be-all. Certainly if you use AWS exclusively you should pay close attention, since I’m sure its innovation in the future will be important and useful.

However, we need an open source, cloud-agnostic solution to this problem. Libraries such as Fog, Boto, libcloud, etc. are not the answer. These libraries are just that… libraries to cloud APIs.

Ideally, we would have a declarative language to define infrastructure (incrementally, as well), and tools to consume this language and use the various libraries noted previously to spin up cloud infrastructure, perhaps even across different cloud providers.

CloudFormation is a brilliant move by AWS and is a technology worth watching, but at this stage its uses are limited, and it leaves space open for an open source alternative, which I hope will come about.

Flexible Local AWS Credentials Management

Fri, 25 Feb 2011 09:23:00 -0800

The Amazon Web Services(AWS) command line tools require certain environmental variables to be set up, such as EC2_HOME for EC2 and AWS_ACCESS_KEY_ID and so on. Most people throw this in their bashrc file and call it good, but this quickly becomes inflexible if you have to manage more and more accounts.

For example, I happen to manage 4 separate AWS accounts (my personal account and 3 other projects).

This post outlines how I manage my credentials in a DRY (Don’t-Repeat-Yourself) and flexible way, such that I could easily add more credentials in the future if I needed to.

Directory Structure

I maintain a very specific directory structure for my AWS related files:

~/.aws
|- bashrc
|- credentials/
|-- setup/
|---- personal
|-- personal/
|- keys/
|-- personalkey.pem
|- ec2/
|- cloudformation/
|- ...

I’ll explain each below:

`bashrc`

This file is the main hook that my main bashrc (at $HOME/.bashrc) sources. It sets up the various enabled AWS modules and also defines the function for selecting AWS credentials.

The basic idea is to consolidate all the global and unchanging AWS related environmental variables into a single bash file, so that it is easy to add new services without cluttering my main bashrc.

You’ll see later that the changing environmental variables (those related to authorization) live in specific credentials files which are enabled using the aws_credentials function.

You can view this file in my dotfiles repo.

To enable it, add a line like this to your main bashrc:

test -f "$HOME/.aws/bashrc" && . "$HOME/.aws/bashrc"

`credentials/`

The credentials directory is in charge of holding all files related to site-specific credentials as well as the bash scripts to setup the environmental variables. There is always a setup/ folder within this directory which contains the bash scripts to toggle different site keys. For example, I may have a personal file in credentials/setup/personal which looks like the following:

As you can see, it simply sets the required AWS environmental variables. Easy! But how do you enable this? Use the aws_credentials function in your terminal, anywhere:

aws_credentials personal

This will find the credentials/setup/personal file and load it for your current shell.

Also important to note, I store all the required files such as the certificates for API calls in a special credentials/SITENAME directory (personal, in this case).

`keys/`

I store all my EC2 private keys in this directory for easy access with SSH. For example, my personal account has a mh_key.pem. To SSH into one of my instances, I just do the following:

ssh -i ~/.aws/keys/mh_key.pem ubuntu@my-ec2-ip.com

I’m still looking to improve this, but it works fine for me now.

`ec2/`, `cloudformation/`, etc.

I extract the command line tools for the various AWS services and place them directly in the .aws directory. I may pull these out further into a services directory or something in the future but for now it works fine. You can see how the various services are enabled in my AWS bashrc (linked to above). Each service has its own way of being enabled, so its easiest to just do it manually for each new service.

Conclusion

My method of credentials management isn’t yet perfect, but its much improved from when I started to a point where its very easy to add new services, add new credentials, change existing credentials, switch sites, etc. I plan on further improving this as time goes on, but hopefully this post is enough to get you started with managing your growing number of AWS credentials.

Let's Get Back to Business

Fri, 21 Jan 2011 21:53:00 -0800

Five years ago, and for about two years after that, I was an avid blogger. I maintained a personal blog that focused on PHP (specifically the Zend Framework) and a year later I launched a successful blog on Erlang, known as spawn_link. Both blogs had a readership of over 500 people, which, while not trying to brag, I was proud of.

Though both blogs brought multiple serious job offers, it was my personal blog which got me my first two jobs in the tech industry (note that I was only 17 when I started blogging!). My first job was at Zend as a contractor doing screencasts for the Zend Framework, and my second job, and still my current employer, was with Citrusbyte, a modern web development shop (which introduced me to the Ruby programming language).

Both jobs kept me busy, super busy, and the result was that I stopped blogging!

The past few years I’ve worked with some extremely smart people, had some unbelievable experiences, and have grown — in many ways — at a rapid speed. There is still so much out there I want to do and things to learn, but I think my experiences the past few years and ongoing deserve to be put down to “ink” for others to gain from.

And that’s my goal with this blog: Let’s get back to business, the business of giving back for everything that I’ve gained.