Jordan Brough

Animoto's 5-word acceptance speech

Jordan — Tue, 09 Jun 2009 15:27:06 GMT

Presenting our CFO:

Do I work for the greatest company on earth or what?

Animoto highlights the Webbys

Jordan — Tue, 09 Jun 2009 15:25:24 GMT

Animoto was asked to create a highlight clip for the Webby's (which we also got to accept an award from) and it turned out really well:

Highlight Reel 2009

Animoto wins a Webby

Jordan — Tue, 05 May 2009 12:15:45 GMT

The awesome team that I'm lucky to be a part of at Animoto has taken home an 'oscar of the internet' in this year's Webby Awards in the Services & Applications category. Go Animoto!

Slow Ruby on Rails tests fixed!

Jordan — Thu, 16 Apr 2009 05:14:24 GMT

I can't thank Barry Hess and Andi Schacke enough for writing and commenting on this post and saving me from all the darkness and despair ;) I've experienced over the past few months as my Rails test times plummetted on this shiny new MacBook Pro. I recompiled as per Andi's comment and my times are now about a fourth what they were. Thank you both.

Revised Hivelogic instructions:

curl -O ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.7-p72.tar.gz
tar xzvf ruby-1.8.7-p72.tar.gz
cd ruby-1.8.7-p72
./configure
make
sudo make install
cd ..

MWRC: Jeremy McAnally - Machine Learning

Jordan — Mon, 16 Mar 2009 14:04:34 GMT

Jeremy McAnally - Machine Learning

Works for entp. His latest app: tweekapp

My Summary

Jeremy was out to convert everyone to doing DSLs the 'right way'. Some salient points were:

the point of a DSL ought to be to make the DSL easier to use for the experts.
'external dsls' (i.e., you make up your own language) can be great for simple project, like config-writing code
'internal dsls' (e.g., ruby-code-dsls) get a lot of stuff for free (loop structures, etc)
dsl composition (using more than one dsl at a time) can cause namespace issues (he didn't really offer much in the way of solutions here as far as I can remember)

DSLs

Used by experts, for specific problems.

Domain

Different problems have different domains, and they each have their own language.

Specific

DSLs that are very generic or look a lot like english don't make sense to him. DSLs are supposed to make the brain-to-code jump much smaller. Also, why waste the keystrokes?

External DSLs

These are DSLs that don't use an existing language (e.g. ruby). Sounds like he doesn't like them as much in general, though there are specific cases where the language structure is simple enough that it makes sense (e.g. puppet).

Internal DSLs

Implemented in an existing language (e.g. ruby). Loops & basic things like that are baked-in and already understood by end-users. External DSLs have to go to a lot of work to create these structures & etc. When to use? Look at the possible complexity of your code, look at what your 'experts' will use.

Examples:

Rails Templates (written by him).
Ambition - 'Take the suck out of SQL'
Foundry (Another of his).

Designing the DSL

Think about the problem domain
Deconstruct -- remove the cruft. DSLs are not fake English. No Finglish allowed.
Implementation

Method chaining

Approach 1: return a new object every time

Approach 2: return self after changing state

Choosing an approach -- think about which is more concise in your case, also think about how #2 can get ugly & unwieldy.

Dynamic dispatch

He likes using closures. Some extra chrome -- if that's bad maybe look at an external DSL.

Compositon - meshing DSLs. Presents namespace issues.

Decoupled DSLs

In his opinion, the DSL ought to be decoupled from the library behind it. The DSL is just the interface to the logic. Thus, when testing you test them separately.

MWRC: Kirk Haines - Vertebra

Jordan — Fri, 13 Mar 2009 21:12:53 GMT

Kirk Haines - Vertebra

Kirk is working for EngineYard on Vertebra

Jordan's Summary

Vertebra is a solution from EngineYard to manage processes in the cloud. Still in development (no production usage so far). Sounds interesting, will probably take a look at their source code if for nothing else than an interesting architecture. Uses XMPP, which I seem keep hearing about these days.

Cloud Computing

Clouds are great b/c it's many, many machines waiting to do you bidding.

Clouds are hard b/c there are many, many machines to manage.

Scripted SSH works, but he says it has its limits. Vertebra built to be 'fault-tolerant operations among autonomous agents'.

Enter Vertebra

Built on XMPP.

very chatty protocol
upside to chatty-ness is fault tolerance. back-and-forth paints the picture of message reception.
goal is: if network dies, operation in progress should be able to pick right up where it left off

For an XMPP Server they use EJabberd

The pieces of Vertebra are agents. E.g. a logger agent.

Special Vertebra Agent: Herault

originally written in ruby, rewritten in erlang for performance & scalability
new agents coming on board advertise their capabilities to Herault
when looking for capabilities ('show me all loggers') you ask Herault
also handles authorization for dispatching operations
when asking Herault to dispatch an operation, can specify :all or :single (more options in the works)

Concurrency API

Talking about using vertebra to schedule future units of work that may or may not execute depending on a given set of conditions.

MWRC: Yehuda Katz - The Great Rails Refactor

Jordan — Fri, 13 Mar 2009 21:13:39 GMT

Yehuda Katz - The Great Rails Refactor

Yehuda is working for EngineYard, was working on merb, is now working on rails.

My Summary

He says he's talking more about the future of rails & less about what-came-from-merb these days.

Yehuda's presentation really seemed to be centered on componentization of the entire Rails stack

Pluggable persistence layers (ActiveRecord, DataMapper, Sequel, CouchDB, etc)
Using and becoming middleware
Making rails' ruby modules less tightly coupled with each other
Replaceable rails components (ActiveRecord, ActiveSupport, Action)

He ended talking about how all these areas of componentization will open up new possibilities of competition (and thus greater health & growth) for the whole Rails ecosystem. An interesting thought in juxtaposed with the elimination of Merb vs Rails competiton. I'm a bit skeptical of whether this can be really pulled off, but I'm excited to see what the brilliant minds behind all this produce.

ORM Agnosticism

Possible in rails right now, but things like form_for ought to work also.

There were two interfaces that merb switched on: ActiveRecord/Datamapper or Sequel. Not too bad but not ideal.

The solution: ActionORM (Lori Holden) - a way of making any ORM look like ActiveRecord.

Two options going forward for compliance for ORMs

mimick AR
provide a proxy that supports the key methods

A few core things will be covered (like form_for) but it will be up to the community to create extensions for the rest. It's hard for them to ascertain what's going to be useful up front and they want to move forward quickly.

Rack

Some middleware they used in merb: PathPrefix, ConditionalGet, Static

Static - can use for tweaking static asset handling for dev mode (and turn off in prod)

Rails middleware usage: Session middleware, Param parsing, Failsafe

This is all working toward amore unix-like setup, requests come in and get routed to pieces (Sinatra, Rails, etc) which are not special -- they just know how to handle the same thing.

Callbacks

He worked on improving callbacks a lot. His first analysis of Rails showed callbacks taking 20% of the time for a request (admittedly a simple request). A bit on the history of callbacks in Rails: there was no unified callback system originally -- lots of different ones. Then it was refactored into callbacks.rb. But it didn't solve all the needs of all the pieces and thus there was a lot of reaching-into-and changing going on that added complexity and slowness. Yehuda went through & made a single calback system that served all needs. Improved actual calling of callbacks - went to 'compilation' instead.

On complex code: Sometimes it's ok to have a complicated bit of code, as long as your abstraction is leak proof & well-abstracted. E.g., nobody cares that ruby is complicated internally because it's done well enough that nobody has to go digging through the C code.

One big problem w/ current rails code -- poorly named (confusing) instance variables make it hard to read & understand.

Upcoming: Abstract controller. Will be module based (truly module based, not tightly coupled modules like rails currently has).

Ruby tip: If you include a module in a class you can super from a module method. Much better than alias method chaining, which can get really messed up if things aren't chained in the right order.

About the plugin api -- there are lots of methods in Rails that are not for everyday use, but are available for plugin authors. These are methods that they don't want to clutter the API docs with, but they will stand by them as far as keeping them working the same way. Sort of a 'hidden' but valid public API. Still thinking about how to provide plugin API for the future of Rails to avoid all the crazy monkey patching that currently goes on.

Upcoming: Debug Toolbar like Django's, but more pluggable.

Orchestra -- gets the pieces working together. Hopefully will make it a lot easier for people to add instrumentation (especially specialized instumentation) to Rails, not only for fiveruns & etc, but for normal end users.

On Competition

Yes, the competition between Rails & Merb is gone. However, Yehuda says that they've really eliminated a less-useful competition in order to open up a more useful one. That is, competition on the component level. The want to make the pieces of Rails compete instead of the entire framework compete. E.g., Sequel vs ActiveRecord, your own ActionController, etc.

MWRC: Jon Crosby - Middleware

Jordan — Fri, 13 Mar 2009 21:13:51 GMT

Jon Crosby - Middleware

getcloudkit.com
contributor to rack-contrib project
working at Engine Yard on Engine Yard Solo

CGI

Remembering the bad old days...

Things we maybe could pull out of our apps

authentication
caching
openid + oauth

Rack

http://github.com/rack/rack-contrib
based on http model of having layers
based on wsgi (web server gateway interace)
Interesting -- Rack::CSSHTTPRequest
Rack::Cache
Check out his cloudkit - intersting out it builds up rack items w/ oauth, openid items & etc
Middleware announcing itself - Middleware can announce it's presence via env headers
Adding sinatra as middleware before Rails

Comment from Yehuda: they are currently working on rails as middleware stuff

MWRC: James Edward Gray, II

Jordan — Fri, 13 Mar 2009 21:13:56 GMT

James Edward Gray, II

If you want to learn Ruby you should read code, lots of code. He estimates that an 'expert' ruby programmer would read more code than he writes.

Examples of good code libraries

Restclient

clean interface to restful services
this is networking code done right
quote he likes: 'Everything in a method should be written to the same level of abstraction'
logs from restclient are executable ruby code

bj

background priority queue for rails

terminator

robust thread/pipe/etc level code written by an expert

slave

trivial multiprocessing with built-in ipc

The art of code reading

when you start reading, relax. not all code sucks & you probably won't understand the author's reasons at first
don't start with Rails (too big & complex)
have a goal
get the source code open in your standard text editor (e.g. use gem unpack or get a copy from a vcs)

Happy Valentines Day!

Jordan — Mon, 16 Feb 2009 03:43:59 GMT

Two videos from our family to yours. :)

Enterprise Ruby

Jordan — Wed, 31 Dec 2008 21:10:53 GMT

(from Maik Schmidt's Enterprise Recipes with Ruby and Rails)

"Back in the mid-90s, an experiment started as a way to make average developers more effective, because the demand continued (as it does today) to outstrip the supply of good developers. If the software industry can ﬁgure out a way to make mediocre developers productive, software development can expand to enterprise scales. Thus, we saw the rise of languages like Visual Basic and Java and later C#. These languages were speciﬁcally made less powerful than alternatives (like Smalltalk). The goal of the Lockdown Experiment: make tools to keep average developers out of trouble while still being able to write code. But then a couple of interesting things happened. First, creating restrictive tools and languages didn’t really keep average developers out of trouble, because average developers sometimes apply great ingenuity to coming up with ridiculously complex solutions to problems. But while this didn’t really make the average developers better, it put a serious governor on the best developers. The whole industry seemed to be optimizing for the wrong thing: safety at the expense of power, with the stated goal of creating software faster. Yet, we didn’t produce software faster; we just annoyed the best developers. The second effect was this new wave of languages was so restrictive that they immediately had to start supplementing them to get real work done. For example, in the Java world, the second version added a bunch of new features (like anonymous inner classes), and eventually some limited metaprogramming was added to Java via aspect-oriented programming.

The real underlying problem with lots of "enterprise languages" is one that Stuart Halloway of Relevance software summed up brilliantly: ceremony vs. essence. Languages that require you to jump through hoops to achieve results are highly ceremonious, whereas languages that make it easy to do sophisticated things are more essential. At the end of the day, you have to solve problems. You want languages and frameworks that lessen the distance from intent to result. Ceremonious languages sometimes make that distance quite far, requiring lots of work that doesn’t really move your solution forward. More essential languages get out of your way, making the distance from intent to result shorter. "

Merry Christmas!

Jordan — Thu, 25 Dec 2008 17:50:16 GMT

"True happiness comes only by making others happy—the practical application of the Savior’s doctrine of losing one’s life to gain it. In short, the Christmas spirit is the Christ spirit, that makes our hearts glow in brotherly love and friendship and prompts us to kind deeds of service." ¹

Extra Flash crossdomain.xml wrinkles

Jordan — Tue, 09 Dec 2008 16:52:12 GMT

Had a lovely time trying to figure out why my flex widget couldn't talk to our sever via https. Turns out that (even with a crossdomain.xml file) a swf served from http cannot access https, unless you add an extra special parameter of 'secure="false"' to the crossdomain file. I really wish flash returned more helpful error messages than 'Security Error'.

http://livedocs.adobe.com/flex/3/html/help.html?content=security2_15.html

We're using this to allow secure communication from our non-https page for some ajax login & fetch behavior. Using the flash widget as a proxy since same origin policy for javascript prohibits just about everything if you need a secure communication w/o having the whole page in https. Ajax requests are prohibited, the script-tag hack doesn't work (login params would have to go (unencrypted) in the url), and iframes suffer from the same problem. Google uses the iframe trick on some of it's pages --

http://www.google.com/shoppinglist?hl=en

(make sure you're not logged in)

but it seems that that only works because they redirect the whole page when successful, which we didn't want to do. Looks like the flash widget will work.

Ruby Closure Surprises

Jordan — Mon, 27 Oct 2008 15:31:37 GMT

Doing some fun stuff with named_scope this morning led me to some surprising discoveries in Ruby's handling of blocks/closures/etc.

In Ruby, I had thought that lambda was supposed to do arity enforcement and that Proc.new wasn't (and also that return behaves differently in Proc's vs lambda's) and that this was the extent of the differences. See here.

However, I was getting a "warning: multiple values for a block parameter (0 for 1)" warning when doing this:

  named_scope :ordered, Proc.new {|o| {:order => o || "created_at DESC"}}

and I then also discovered that the following also worked (with the same warning) whereas I thought it would have raised an exception because it was lambda:

  named_scope :ordered, lambda {|o| {:order => o || "created_at DESC"}}

My coworker Jeff found this fascinating and revealing investigation into Ruby closures where he reveals "four types of closures and near-closures, expressible in seven syntactic variants." Yikes.

However, I think the chart he has about half way down is either outdated or a little wrong. Here's what I came up with after a little investigation of my own. Perhaps I'll write down a test for these at some point to test future Ruby versions (and to make sure I got the chart right). But for now --

                                                    "return" returns from closure
                                   True closure?    or declaring context...?         Mismatched arity behavior
                                   ---------------  -----------------------------    -------------------
1. block (called with yield)       N                declaring                        Behavior 1
2. block (&b => f(&b) => yield)    N                declaring                        Behavior 1
3. block (&b => b.call)            Y except return  declaring                        Behavior 1
4. Proc.new                        Y except return  declaring                        Behavior 1
5. proc                                    <<< alias for lambda in 1.8, Proc.new in 1.9 >>>
6. lambda                          Y                closure                          Behavior 3
7. method                          Y                closure                          Behavior 2

* Behavior 1: Succeed (supply nil for missing args and ignore extra args).  Exception: When block is defined to accept exactly 1 parameter then warn on any mismatch
* Behavior 2: raise ArgumentError on mismatch 
* Behavior 3: same as Behavior 2 except for the following:
    - Same as Behavior 1 if block has exactly one parameter, e.g. -- lambda{|a| puts a}.call # warn and succeed
    - Same as Behavior 1 if block was defined w/o argument pipes, e.g. -- lambda{puts 'hi'}.call(1,2) # succeed

It's a crazy Ruby world :)

Creating Cross Platform Windows and Mac Installer CDs

Jordan — Mon, 31 Mar 2008 14:36:28 GMT

I'm working on an Adobe AIR cross-platform app at work. We're providing a physical CD for distributing the application and I wanted the CD to work like a real Mac CD should look like -- Background image in the finder, hide all extraneous files, auto-open the finder window when the CD is inserted, etc.

Getting just this to happen requires some trickery, and getting it to happen while also putting an auto-launching Windows installer on the same disc proved to be even more of a game.

After a lot of googling I think I've got a pretty good process down and wanted to share, so here are the details.

Core Concepts

The standard ISO CD filesystem doesn't let you do all the good stuff that you'll want to do for Mac (finder background image, hiding files, etc). You need to use some variant of HFS. However, Windows can't read HFS.
'Hybrid CDs' solve the previous problem. From what I understand, a hybrid CD contains an HFS volume and a standard ISO volume. Windows machines can't see the HFS volume and OS X machines ignore the ISO volume when there is an HFS volume present.
You'll need the OS X Developer tools for these steps
From what I understand, Roxio Toast 7 & above make this much easier, but this is more fun and it's free. :)
It is much easier to test all this out in Windows if you have Parallels (or presumably Fusion) because Parallels let's you mount an iso in Windows as if it were a CD, so you don't have to waste your precious blank CDs creating actual copies. :)

Steps

Create a standard folder on your harddrive and put all the install files (windows and mac, including an autorun.inf for your Windows installer, presumably) in it. We'll call it 'install folder' for now.
Make a user-friendly finder window

To make a nice OS X finder window for a CD (or dmg or whatever) we want to have a friendly background image that helps the user understand what they're supposed to do. Go grab AdiumX to see an example of what I mean. We have to include this image on the CD, but we also want to hide it from the end user so that it doesn't clutter up our install window. Here are the steps:
1. Create your background image.
2. Create a folder called 'background' inside 'install folder'.
3. Put your background image into this folder.
4. Use terminal to rename this folder to '.background'. e.g.: mv background .background (The folder should then disappear from view in the Finder)
5. Open 'install folder' in the Finder and hit Command+J. Make sure that 'This window only' is selected. Click 'Picture' for the background. In the dialog that comes up, hit command+shift+g to bring up a box where you should enter the path to your '.background' folder. This allows us to navigate there even though it is hidden. Select your background image and click OK.
6. Click 'Picture' again for the background. For some reason it gets reset after selecting the picture.
7. For each file that you'd like hidden in both windows & mac (e.g. perhaps the window autorun file), run this command in Terminal:
  
  /Developer/Tools/SetFile -a V /path/to/your/folder/name-of-file-to-hide
Get rid of the finder toolbar (click that top-right oval button) and resize your window & arrange your icons, leaving the windows icons clumped together somewhere for now (we'll hide them in a minute)
Create the hybrid ISO using hdiutil Run this command in Terminal to turn your folder into a hybrid ISO:

hdiutil makehybrid /source/folder/name/ -o outputfile.iso
Mount the ISO in read-write mode: Run this in terminal:

hdiutil attach -readwrite outputfile.iso
Make files invisible to Mac ONLY (e.g., your windows installer file) by setting the invisible bit on them: Again in terminal:

/Developer/Tools/SetFile -a V /path/to/your/folder/name-of-file-to-hide
Make the finder window auto-open when the cd is inserted/image is mounted: In Terminal:

sudo bless -folder "/Volumes/discName" -openfolder "/Volumes/discName"
Test it out. Mount your ISO and unmount it a couple times to see the wonderous installer. If you have Parallels you can try it out in Windows too by using 'Connect Image' to simulate it as a CD in Windows
Open Disk Utility & drag your iso file into the left-hand pane. Click 'Burn' in the toolbar above. Insert a blank CD and let it burn.

Done! Try inserting it into a Mac and a Windows machine and be amazed by the beauty of it all.