My take on: How to give your new project a good starting point
So, you have a new shiny project on your lap, so now what? Today everyone is talking about “agile”, and how its better then the old “waterfall” methodology of doing things
-
Waterfall
The “Waterfall” approach is notorious in the software community, some highlights of why this is the case:
-
-
It assumes that the initial requirements will not change
-
There is only one version delivered to the stakeholders, at the end of the process
-
All the planning and “thinking” happen only in the beginning of the project
-
-
Agile
Everyone likes “Agile”, its the “cool” way to manage a project, it asks for less paperwork, less work upfront, built from the ground up to support changes, to name a few conceptions about it. While I call it “Agile” here, in reality it is a world of sub cultures, some of which are: “XP” (eXtreme Programming), “scrum”, “Kanban” to name a few. The difference between specific cultures is not always clear, we don’t necessarily have to stick to one, we can mix and match, like I am doing here, thus from this point onward I will just ignorantly call it “Agile” with out specifying the specific methodology. The main strengths going for the Agile methodology are
-
-
Short upfront work phase (requirements, design ..)
-
Iterations - project is broken down to deliverables, which is translated to real value delivered very early in the project lifetime
-
Changes are welcome and expected and are introduced into following iterations (bugs found in previous deliverables, design “inadequacies”)
-
-
Doing things in small pieces is great, just ask a WEB developer, how there’s nothing quite like writing a bit of code, refreshing the page and seeing the changes reflected, especially when working on a big complex project. This gives you a feeling of progress and achievement, even though the end goal is far far away..
-
Some Agile approaches talk beyond writing business code and going into proactive bug searching using TDD (Test Driven Development), which turns out to be a good fit to the overall process as it secures quality of delivered pieces of logic. BDD (Behavior Driven Development) is very similar to TDD, but instead of going after “test coverage” it takes on a higher level, business specification driven approach, where the business embodied user stories are part of the actual test suite.
-
My recipe for success
-
User Stories
-
Mock-ups (in the case of an app that has UI)
-
HLD (High Level -system component- Design)
-
Process Sequence diagram / Workflow diagram (possibly atop HLD)
-
-
Tooling
Tools that keep me productive:
-
-
Google Document (User Stories)
-
Google Drawing (mockups, diagrams, HLD)
-
Lucidchart (diagrams)
-
-
In addition, I use a light weight web based project management tool as a central place to keep reference to important docs and user stories, and see my progress (and others working on the project). A great tool for the job is Trello.com - which free and super simple!
from the list select the Android version your afer, right now its Android 4.0.3 (API 15) (I also installed Android 2.3.3), click on Install packages to get the installation started
-
-
setup your env echo "export ANDROID_HOME=~/Applications/android-sdk-linux/" >> ~/.bashrc
-
-
refresh your currect session source ~/.bashrc
-
-
-
Adding SBT support to IntelliJ
-
open File > Settings > Plugin
-
-
click on Browser repositories
-
-
find SBT, right click and select Download and install
create folder ~/bin/ (if you dont have ~/bin in your PATH, then you can do this: echo "PATH=$PATH:~/bin" >> ~/.bashrc (refresh your currect session source ~/.bashrc)
Finally, now that you have all the tools you need, proceed to creating a new project
-
run the android plugin project setup tool: ~/bin/g8 jberkel/android-app, follow the onscreen questions (or press return for defaults) this will create the project folder and create the files needed to build an android project
“User Story” is an important communication tool, it is used between you (the developer) and who ever is making the requirements (the stakeholder). A “contract” which (unlike one written by a lawyer) is clear, understood and agreed upon. On our side (development) user stories embody a work unit that can be further broken down and estimated.
-
Key value points:
-
-
Language - its key that user stories are written in a language everyone can understand (stakeholders, developers etc..)
-
Perspective - user stories need to be written from the perspective of the “feature user”, so they represent real world value
-
Estimation - to help you get better and more precise estimation for the project overall
-
Prioritization - what needs to be delivered soon/later
-
-
Lets imagine we just got a call from the product manager, where he is telling us about this new survey tool:
-
It needs to collect answers from users via a link sent to our users by e-mail
-
It needs to be summarized and sent to … once a month on the 1st
-
Managers need to be able to see past surveys
-
-
Hmm.. OK, thats pretty clear, but suppose this can be written clearer, lats try using the following template: (not my invention;)) As a [role], I want [Requirement / Feature], So that [Justigication/ Reason]
-
The requirements now look like this:
-
-
As an end user, I will be able to follow a link I got via e-mail, so that I can fill a survey and express my opinion
-
As a manager, I will receive an e-mail with a digest of surveys, so that I can evaluate performance
-
As a manager, I will be able to access old surveys, so that I can compare and evaluate performance over time
-
-
By reading the above I can clearly understand the required functionality and the motivation behind writing it, this gives me a good starting point. Since I am the one to implement this, I will need to provide estimates, so that product can plan ahead and communicate the product road map onward. Unfortunately, “The Devil is In The Details” as they say, so giving estimates based on points 1,2,3 is still difficult. Points 2 & 3 are “huge” stories, also known as “epics”, we need to further break down these stories, while still using the same language and practices:
-
-
As an end user, I will receive an email with a link as a result of some communication with a company, so that I can follow it and fill a survey
-
As an end user, I get a set of questions to answer, so that my opinion is collected
-
As a manager, I will receive an e-mail with a digest of surveys, so that I can evaluate companies performance
-
As a manager, I will have a survey management portal, so that I can overview previous results and compare to
-
As a manager, I can select a specific survey, so that I can extract meaningful actionable information
-
-
The functionality expected of deliverables from the points above is clear, and now I feel more comfortable estimating the time it will take me to complete each. Along with the estimation, these can now be prioritized against each other and against other ongoing projects..
-
Summery
The beginning of a new project is the perfect time to introduce user stories, it will help with getting the conversation going, further clarifying the scope and requirements. While the project scope might change, user stories are (optimally) modular, decoupled and encapsulated, this should help with managing changes and their effect on the overall project.
System Architect - mostly interested in the software stack, deployment and the database design
-
Team Members - mostly interested in the more lower level stuff like modules and API’s
-
Other Teams - mostly interested in API’s
-
-
Feedback on design is great for everyone involved, setup meetings with relevant parties to do DR (design review)
-
UML and Other Vegetables
UML is complex, you don’t need to know it perfectly, start with what makes sense, and slowly pick up tutorials. Remember, its a visual tool that helps you communicate, not everyone who will look at your designs has the time (or cares for) to read throughly a book about UML and its specifics. In Einstein’s words “Make everything as simple as possible, but not simpler.”
-
Disclosure
Design does not last. Nobody is going to update the design as the system grows and evolves. Having said that, can still be used a good reference.
“Strong typing is for people with weak memories” - not sure who originally said that, but you have got to love the enthusiasm people have for dynamic languges. What is there not to love? powerful basic language constructs such as closure, prototypal inheritance, objects as first class citizens and anonymous functions, mainly, with which you can simulate almost anything static languages have to offer!
-
Proof you say? no problem! here are few implementations that are worthy of noting:
(if you have other interesting examples let me know!)
-
Convention over configuration
The combination of CoffeeScript’s “classical” Class like system and Backbone.JS’s MVC style structure, give (IMHO) the best value in terms of complexity and pure fun writing apps. The challenges I come across while building large complex applications is the inherited nature of a true dynamic language “typelessness“.
-
The ugly monster, that is -a large application written in JavaScript, pushed me into struggling with these two main challenges:
-
-
Many-to-many PubSub communication
-
3rd party API’s
-
-
Many-to-many PubSub communication
While the PubSub (also known as observer pattern) in JavaScript is a powerful tool to decouple your code, having multiple callers and listeners on the same event makes it really difficult to keep track of the actual common API for these events. Lats try to reflect on this while looking at an actual implementation.
-
(please excuse my CoffeeScript)
-
Pub/Sub implementation in Backbone/Underscore.JS
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14
pubsub = _.extend {}, Backbone.Events pubsub.on 'myEvent', (param1, param2)-> x1 = param1 + param2 # do something with x1
# in some other obscure location in the code pubsub.on 'myEvent', (param1)-> x2 = param1 # do something with x2
The “pubsub.on(event name, callback)” API is very common amongst different JS pub/sub frameworks. here’s an example of how a generic implementation might look like:
If your using the very popular Socket.IO library, for realtime client/server communication, you are using the exact same API, in which case it makes it even more complicated to keep track of, since the ambiguity is manifested on both the client and the server.
-
3rd party API’s
This is how a type related bug might look like in a dynamic language
-
1 2 3
a = "42" # returned from some API a == 42 => false
I’m introducing a convention here, that helps keep the API consistent across the application, the approach here is to create wrapper functions, that are used as both the API documentation and run-time validation facility for required and optional arguments.
-
Example:
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
# declare the events and their APIs Messages = { LoginEvent : Contractor.Create( "LoginEvent", Contractor.Required("user ID"), Contractor.Optional("additional info") ) AppErrorEvent : Contractor.Create( "AppErrorEvent", Contractor.Required("user ID"), Contractor.Required("error description"), Contractor.Optional("exception") ) }
# register on event API pubsub.on Messages.LoginEvent, (userID, info)-> #... do stuff here pubsub.on Messages.AppErrorEvent, (userID, description, exception)-> #... do stuff here
# trigger the event pubsub.apply(pubsub, Messages.LoginEvent(112233, "mobile")) pubsub.apply(pubsub, Messages.LoginEvent(112233))
pubsub.apply(pubsub, Messages.AppErrorEvent(112233, "user did not have access to resource", e)) pubsub.apply(pubsub, Messages.AppErrorEvent(112233)) # since we did not provide the second required argument, this will log an error and return null
-
With NodeJS, I am using the same code on the front and back end, which makes it really easy to manage my Socket.IO communication.
-
If you have been paying attention, there’s still the type issue. Since I work allot with Backbone, it made a lot of sense to write “backbone-typed” which adds optional typing to Backbone-Models. Example:
Contractor is a factory of contracts, that helps with documenting our APIs. The child_process facility built into node is using EventEmitter (v1), and does not support callbacks, so it is a great fit for Contractor! Since I needed it for a distributed worker framework I am working on, I added a helper facility called “Lawyer”, it basically reads the contracts and routes them to the provided object.
-
Let us take the parent and child example code from Node’s official documentation (http://nodejs.org/api/child_process.html) and see how simple it would be to add some documentation.
Node’s child_process facility provied us with a very basic means of communication, we simply call n.send({any: "thing"}) and it automagically recieved by the forked child. Let’s spice it up with a more RPC message passing style, with contracts and lawyers
var contracts = require('./contracts'); var Contractor = require('contractor').Contractor; var Lawyer = require('contractor').Lawyer; var cp = require('child_process');
Unlike the classic FIFO queue, jobs are scheduled for execution in the future, possibly on a specific day, week, month year.. similar to “cron” in a Unix OS. Unlike the normal Unix “cron” implementation, we want it to be distributed, so that several machines(workers) can query and then execute these jobs.
-
Implementing
-
Planning - The standart Unix cron format is very flexible and has been time tested for describing schedules
-
Managing/Coordinating - The queues are stored in Redis Sorted Sets, Lua will be used for atomic execution
-
Execution - Getting the pieces together
-
-
Planning
“Unix Time” format, is the number of milliseconds since (UTC) 1 January 1970 Example (* the actual unix time number is in UTC, while the formatted date is localized)
-
1350745504313 == Sat Oct 20 2012 17:05:04 GMT+0200 (IST)
-
Using this format will make it possible to slice the queue with specific point in time in mind.
-
Managing/Coordinating
Redis’st will be holding the job IDs as members scored by their corresponding execution times(in unix time format).
Finding a job to run: ZRANGEBYSCORE cron:queue 1350745504314 LIMIT 0 1
-
-
Execution
Without going into any platform specific implementation details, this is what the process looks like
-
-
When a new schedule is added, jobs are created for upcoming execution times, which are calculated for the next two hours and added to our cron queue
-
A special job is added to be run every hour, it will run over all the schedules, calculate and add their execution time to the scheduled queue, two hours into the future
-
Workers will be polling the database every second to find jobs
-
Once an entry comes up, it is removed from the queue and added to another Sorted Set queue, so we dont “forget” any jobs..
-
-
As for items 3 and 4 above, it is not possible to implement this logic in the client, because for example when we find a job that needs to be executed, another worker might grab it, or worse they both “grabbed” it and are now working on the same job. To work around this, we need to write a little Lua script, since Lua scripts are executed serially, just like any other Redis command
local currentTimeMS = KEYS[1] local workingQueue = "scheduler:working" local scheduleQueue = "schedule" -- for easier reading, epoch time is in milliseconds local second = 1000
localfunctionfoundJob(jobID) -- implement removing the job from all other queues -- implement updating job status where the actual job object is stored end
localfunctionfindNextJob() local result -- check if there are any "stale" jobs result = redis.call( 'ZRANGEBYSCORE', workingQueue, 0, currentTimeMS-60*second, 'LIMIT',0,1 ) if result[1] ~= nilthenreturn foundJob(result[1]) end -- check the normal queue result = redis.call( 'ZRANGEBYSCORE', scheduleQueue, 0, currentTimeMS, 'LIMIT',0,1 ) if result[1] ~= nilthenreturn foundJob(result[1]) end -- if we didnt find anything return nil returnnil end
return findNextJob()
-
And there you have it! a “Distributed scheduled queue” implemented on top of Redis.
-
If you are going to implement this with node, I created a little cron format parser you can find here: JSCron
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/2012/11/14/making-the-case-for-kotlin/index.html b/2012/11/14/making-the-case-for-kotlin/index.html
deleted file mode 100644
index 6969604..0000000
--- a/2012/11/14/making-the-case-for-kotlin/index.html
+++ /dev/null
@@ -1,305 +0,0 @@
-
-
-
-
-
- Making The Case For Kotlin | Uniformlyrandom
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
The Kotlin project page has this to say about Kotlin: “is a statically typed programming language that compiles to JVM byte codes and JavaScript.” project page
-
Kotlin is a very interesting project, I wanted to check it out, so I decided to write this blog post exploring some design aspects behind Kotlin, and specifically how it compares to the leading JVM “hacker” language - Scala.
-
Who Is Behind Kotlin?
Even though there are several decent free IDE’s available today (Eclipse, NetBeans, Visual Studio Express..), it (could be) surprising that JetBrains is able to rack in between 199$ for a personal license and up to 699$ for a commercial license (for companies) including 1 year update subscription (I have indeed bought one personal license for my development needs). If you are developing on top of VisualStudio you are (most) probably familiar with the 250$+ plugin ReSharper.
-
I would argue that this makes them a good candidate, with insight into good and bad language features, but lets not get ahead of ourselves and set the expectations too high.
-
Why Another JVM Language?
JVM is a well known platform, its weaknesses and scalability strategies are available and documented well on the web, its being used as classic Java run-time VM and for some other popular languages like Scala, Groovy, Clojure and others. Many modules have been already written in Java, normally JVM run languages are compatible with the existing Java ecosystem.
-
Why Static Typing?
In the old (and still ongoing) battle of dynamic and static languages, the consensus is that dynamic languages are faster and easier way to get things done, on the static languages side they allow programmers harness the power of intelligent IDEs to scale up and maintain their code. Type-inferred languages try to find a middle ground approach so that you don’t need to write so much boilerplate code (cue JAVA) to support the static typing, while having the benefits of a full compile time type enforcement.
-
Finally, Kotlin Vs. Scala - Fight!
Scala today is the fastest growing JVM language and is probably competing for the same crowd as Kotlin, Kotlin put up a comparison page link so we will simply go over the omissions part and try to make sense of it.
-
OMITTED - Implicit conversions, parameters
What is it? Scala implicits is (IMHO) a single biggest weakness in the language, once they are used, along with the ability to boast symbols for syntax, it quickly stops being human readable and more like a DSL. Which requires many trips to the docs or even worse - the source code.
-
SBT (Scala Build Tool) is the Maven/Ant of the Scala world link, its very difficult to get things done with it without many trips to google/documentation
Lift Is a popular web framework, here’s a snippet I took from one of its documentation pages link (see the "#result *" #> x part)
-
1 2 3 4 5 6 7 8 9
defrender= inputParam match { caseFull(x) => println("Input is: "+x) "#result *" #> x case _ => println("No input present! Rendering input form HTML") PassThru }
-
Thoughts - Indeed for many people this is the part that makes Kotlin a better “scalable” language, by not allowing programmers and libraries hinder the code unreadable without an IDE
-
OMITTED - Overridable Type Members
What is it? Type members in Scala allow you to manage local generic types by assign a type to a class/abstract class/trait member
-
1 2 3
abstractclassHasTypedMember{ typetypeMe }
-
An extending class will provide a specific type for this type type member
-
1 2 3
classExtendingClassextendsHasTypedMember{ typetypeMe= Int }
-
Thoughts - Frankly, its not really clear why Kotlin decided to omit having typed members, I could be missing something.. Never the less Kotlin usually takes the minimalistic and simplistic route, my guess is that if this is indeed useful, the door is open to add it later..
-
OMITTED - Path-dependent types
What is it? Pointing to a class under another class in Java would mean a specific type (Parent.Child), well not so in Scala. Scala makes the nested class of two same class instances different, much like you would expect two members of two different instances of a single class. Confused? here is some code to explain this better
-
Java:
-
1 2 3 4 5 6 7 8 9 10 11 12
publicclassOuter{ publicclassInner{} publicvoiduseInner(Outer.Inner inner){...} } //.. somewhere in the code o1 = new Outer() o2 = new Outer() i1 = new o1.inner() i2 = new o2.inner() // this would work fine o1.useInner(i2) // works o2.useInner(i1) // works
-
Scala:
1 2 3 4 5 6 7 8 9 10 11 12
classOuter{ classInner{} defuseInner(inner : Inner){...} } //.. somewhere in the code.. val o1 = newOuter val o2 = newOuter val i1 = new o1.Inner val i2 = new o2.Inner o1.useInner(i1) // works fine o2.useInner(i2) // works fine o1.useInner(i2) // THROWS type mismatch exception!
-
Thoughts - It makes some sense that the path to a type that can only exist in an instance is enforced as such.. It would be interesting to know why Kotlin decided to implement this feature in a similar way to Java (conservative?)
-
OMITTED - Existential types
What is it? This is yet another trade-off for Scala to be able to inter-op with Java Let’s see an example:
-
1 2 3 4 5
// Create a case class with generic type caseclassZ[T]( blah:T) // create an instance and cast it to a type created with existential type annotation Seq( Z[Int]( 5 ), Z[String]("blah") ) : Seq[Z[X] forSome { typeX>: StringwithInt }] res1 = Seq[Z[_ >: StringwithInt]] = List(Z(5), Z(blah))
-
Thoughts - Scala supports type variance to describe generic type bounds, Scala designer (Odersky) chose to add existential types support due to three main reasons link: erasure, raw types (lack of generics) and Java’s “wildcard” types.
-
Kotlin took the more “purist” approach (also known as reified types) of dropping support for erasure, raw types, wildcard types and finally existential types.
-
OMITTED - Complicated logic for initialization of traits
What is it? Traits are similar to abstract classes, except that you may use several traits on the same class at the same time (like interfaces).
-
In Scala, traits are constructed before the extended classes
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14
traitAverager{ val first : Int val second : Int val avg : Int = first / second } // if we were to naively construct a class like so: classNaiveextendsAverager{ val first = 20 val second = 10 } // create instance > newNaive() java.lang.ArithmeticException: / by zero // what happened was that our val "avg" was computed at construction time before seeing the values we put in Naive
-
Alternatively the trait can be used with a pre-initialized anonymous class val res = new { val first = 20 ; val second = 10 } with Averager, also the val which depends on other vals (avg in our case) can be made “lazy” with the (obvious) lazy keyword lazy val avg : Int = first / second.
-
As a side note, using def for “avg” could work, but it would mean evaluating whatever is defined under “avg” every-time this method is called.
-
Thoughts - Traits in Scala can have state which could be made tricky. If you read this blog post link you will see Kotlin developers thoughts and a nice insight into what went into designing the inheritance in Kotlin. (don’t forget to read the comments!)
-
OMITTED - Custom symbolic operations
What is it? Scala does not have a built in handling of symbols (like =,+,/ etc), they are actually method calls, as you can have symbols used for method names.
-
Thoughts - This may sound useful at first, but can and (arguably) will create very confusing looking code, like we saw before in the Lift and SBT examples.
-
OMITTED - Built-in XML
What is it? Exactly as it sounds, Scala built in XML native support, so you can do stuff like this
-
1
val holdsXml = <outer><inner>some text</inner></outer>
-
Thoughts - Having a built in support for XML as a language feature is interesting, and sometimes useful. An interview link with Odersky reveals that he would consider removing the native XML support in Scala if had written Scala now.
-
Bottom Line
Kotlin is an idiomatic language that draws from the many good features in todays leading languages, specifically from Scala which is a leading language on the JVM platform today. We’ve looked into omissions on Kotlin’s part versus Scala
-
I’ve very much enjoyed writing this blog post and learning about the evolution that the design of Java/Scala/Kotlin languages, and the choices the respective authors made along the way.
-
To get a better feeling of Kotlin I might pick it up for a small project, at which point I will surely blog about it some more..
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/2014/06/14/Using-Spray-to-mock-3rd-party-APIs-in-your-tests/index.html b/2014/06/14/Using-Spray-to-mock-3rd-party-APIs-in-your-tests/index.html
deleted file mode 100644
index 6de2ec4..0000000
--- a/2014/06/14/Using-Spray-to-mock-3rd-party-APIs-in-your-tests/index.html
+++ /dev/null
@@ -1,266 +0,0 @@
-
-
-
-
-
- Using Spray to mock 3rd party APIs in your tests | Uniformlyrandom
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Using Spray to mock 3rd party APIs in your tests
-
-
-
-
-
-
-
-
If you need to test/mock 3rd party API’s / a crawler / external web service, and you dont want to use some fancy DI framework (I hate the magic in DI frameworks, I use constructor DI instead..), anyway, here’s a helper class I put togather to that end
-
If you haven’t got a chance to try out Spray, you totally should, its a high performance web server built on top of Netty and Akka
-
Spray has a very nice HTTP API. Few things I liked:
-
-
HTTP method mapping spray.http.HttpMethods - GET, POST etc..
-
HttpRequest and HttpResopnse are basically wrappers around data you would expect to have in a request and response data
-
spray.http.Uri wrapper class around what you would expect to find in the URI
-
it really is just an Akka Actor, with just the right amount of control exposed and the rest well hidden behind its nice APIs
import akka.actor._ import akka.io.IO import java.util.concurrent.atomic.AtomicInteger import spray.can.Http import spray.http.{Uri, HttpMethod, HttpResponse, HttpRequest} objectHelpers{ lazyimplicitval testSystem = ActorSystem("helpers-test-system") // used to track the last assigned port val lastPort = newAtomicInteger(20000) // a single expected triplet of method+uri+body typeReplay= (HttpMethod,Uri.Path,String) // factory method to get a replater to play around with defwebReplayer(replay : List[Replay]) : (ActorRef,Int) = { // everytime the factory is used a new port is assigned, // so many instances can co-exist and tests can run in parallel without getting // on eachother's toes val newPort = lastPort.addAndGet(1) // instantiate a new Spray server. Yes, its that simple! val newServer = testSystem.actorOf(Props(newHelpers.ReplayerActor(replay))) IO(Http) ! Http.Bind(newServer, interface = "127.0.0.1", port = newPort) // return a pair of the server instance and the assigned port // the newPort should be used by client code to construct the remote URI with (newServer,newPort) } // object to use to tell the server to shutdown.. objectShutTestServer // a concrete implementation for a Spray Actor to listen to incoming HttpRequest(..) messages // provided a list of messages to be matched against arrived and content to be sent back classReplayerActor(var replay :List[Replay]) extendsActor{ var ioServer : ActorRef = null overridedefreceive: Receive = { case _ : Http.Connected => sender ! Http.Register(self) ioServer = sender() // this is basically a "catch all" HttpRequest pattern caseHttpRequest(method, uri,_,_,_) => // a test to see that this is indeed expected to happen at this point if (!replay.isEmpty && replay.head._1 == method && uri.path == replay.head._2){ sender ! HttpResponse(entity = replay.head._3) replay = replay.tail } else { sender ! HttpResponse(status = 400, entity = "was expecting a different request at this point") } caseShutTestServer => ioServer ! PoisonPill case _ => //ignore } } }
-
And now a cocrete client code example to use this tool
// some random content I would like to receive in this scenario (or test..) val content = s""" |<html> |<head> |<title>sometitle</title> |</head> |<body>some body</body> |</html> """.stripMargin // we use the factory method to get an instance of "replayer" to play around with val (_testWebServer,port) = Helpers.webReplayer( List( (GET,Uri.Path("/somesite"),content) ) ) // and where ever in your actual code, use the port we got back above and have it "hit" the server val whatEver = YourHTTPGetter.fetch(s"http://127.0.0.1:$port/somesite") assert(whatEver == content)
-
Profit!
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..bc06f5b
--- /dev/null
+++ b/README.md
@@ -0,0 +1,4 @@
+blog
+====
+
+zblog
diff --git a/_config.yml b/_config.yml
new file mode 100644
index 0000000..f4a669b
--- /dev/null
+++ b/_config.yml
@@ -0,0 +1,93 @@
+# Hexo Configuration
+## Docs: http://hexo.io/docs/configuration.html
+## Source: https://github.com/tommy351/hexo/
+
+# Site
+title: Uniformlyrandom
+subtitle:
+description: The land where entropy can not predict the expected stream of random thoughts
+author: Roman Landenband
+email: roman.land@gmail.com
+language:
+
+# URL
+## If your site is put in a subdirectory, set url as 'http://yoursite.com/child' and root as '/child/'
+url: http://uniformlyrandom.com
+root: /
+permalink: :year/:month/:day/:title/
+tag_dir: tags
+archive_dir: archives
+category_dir: categories
+code_dir: downloads/code
+
+# Directory
+source_dir: source
+public_dir: public
+
+# Writing
+new_post_name: :title.md # File name of new posts
+default_layout: post
+auto_spacing: false # Add spaces between asian characters and western characters
+titlecase: false # Transform title into titlecase
+external_link: true # Open external links in new tab
+max_open_file: 100
+multi_thread: true
+filename_case: 0
+render_drafts: false
+post_asset_folder: false
+relative_link: false
+highlight:
+ enable: true
+ line_number: true
+ tab_replace:
+
+# Category & Tag
+default_category: uncategorized
+category_map:
+tag_map:
+
+# Archives
+## 2: Enable pagination
+## 1: Disable pagination
+## 0: Fully Disable
+archive: 2
+category: 2
+tag: 2
+
+# Server
+## Hexo uses Connect as a server
+## You can customize the logger format as defined in
+## http://www.senchalabs.org/connect/logger.html
+port: 4000
+server_ip: 0.0.0.0
+logger: true
+logger_format:
+
+# Date / Time format
+## Hexo uses Moment.js to parse and display date
+## You can customize the date format as defined in
+## http://momentjs.com/docs/#/displaying/format/
+date_format: MMM D YYYY
+time_format: H:mm:ss
+
+# Pagination
+## Set per_page to 0 to disable pagination
+per_page: 10
+pagination_dir: page
+
+# Disqus
+disqus_shortname: romanskys-blog
+
+# Extensions
+## Plugins: https://github.com/tommy351/hexo/wiki/Plugins
+## Themes: https://github.com/tommy351/hexo/wiki/Themes
+theme: landscape
+exclude_generator:
+
+# Deployment
+## Docs: http://hexo.io/docs/deployment.html
+deploy:
+ type: git
+ root: /
+ repo: git@github.com:uniformlyrandom/blog.git
+ branch: master
diff --git a/archives/2012/02/index.html b/archives/2012/02/index.html
deleted file mode 100644
index 4025efc..0000000
--- a/archives/2012/02/index.html
+++ /dev/null
@@ -1,436 +0,0 @@
-
-
-
-
-
- Archives: 2012/2 | Uniformlyrandom
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
“User Story” is an important communication tool, it is used between you (the developer) and who ever is making the requirements (the stakeholder). A “contract” which (unlike one written by a lawyer) is clear, understood and agreed upon. On our side (development) user stories embody a work unit that can be further broken down and estimated.
-
Key value points:
-
-
Language - its key that user stories are written in a language everyone can understand (stakeholders, developers etc..)
-
Perspective - user stories need to be written from the perspective of the “feature user”, so they represent real world value
-
Estimation - to help you get better and more precise estimation for the project overall
-
Prioritization - what needs to be delivered soon/later
-
-
Lets imagine we just got a call from the product manager, where he is telling us about this new survey tool:
-
It needs to collect answers from users via a link sent to our users by e-mail
-
It needs to be summarized and sent to … once a month on the 1st
-
Managers need to be able to see past surveys
-
-
Hmm.. OK, thats pretty clear, but suppose this can be written clearer, lats try using the following template: (not my invention;)) As a [role], I want [Requirement / Feature], So that [Justigication/ Reason]
-
The requirements now look like this:
-
-
As an end user, I will be able to follow a link I got via e-mail, so that I can fill a survey and express my opinion
-
As a manager, I will receive an e-mail with a digest of surveys, so that I can evaluate performance
-
As a manager, I will be able to access old surveys, so that I can compare and evaluate performance over time
-
-
By reading the above I can clearly understand the required functionality and the motivation behind writing it, this gives me a good starting point. Since I am the one to implement this, I will need to provide estimates, so that product can plan ahead and communicate the product road map onward. Unfortunately, “The Devil is In The Details” as they say, so giving estimates based on points 1,2,3 is still difficult. Points 2 & 3 are “huge” stories, also known as “epics”, we need to further break down these stories, while still using the same language and practices:
-
-
As an end user, I will receive an email with a link as a result of some communication with a company, so that I can follow it and fill a survey
-
As an end user, I get a set of questions to answer, so that my opinion is collected
-
As a manager, I will receive an e-mail with a digest of surveys, so that I can evaluate companies performance
-
As a manager, I will have a survey management portal, so that I can overview previous results and compare to
-
As a manager, I can select a specific survey, so that I can extract meaningful actionable information
-
-
The functionality expected of deliverables from the points above is clear, and now I feel more comfortable estimating the time it will take me to complete each. Along with the estimation, these can now be prioritized against each other and against other ongoing projects..
-
Summery
The beginning of a new project is the perfect time to introduce user stories, it will help with getting the conversation going, further clarifying the scope and requirements. While the project scope might change, user stories are (optimally) modular, decoupled and encapsulated, this should help with managing changes and their effect on the overall project.
from the list select the Android version your afer, right now its Android 4.0.3 (API 15) (I also installed Android 2.3.3), click on Install packages to get the installation started
-
-
setup your env echo "export ANDROID_HOME=~/Applications/android-sdk-linux/" >> ~/.bashrc
-
-
refresh your currect session source ~/.bashrc
-
-
-
Adding SBT support to IntelliJ
-
open File > Settings > Plugin
-
-
click on Browser repositories
-
-
find SBT, right click and select Download and install
create folder ~/bin/ (if you dont have ~/bin in your PATH, then you can do this: echo "PATH=$PATH:~/bin" >> ~/.bashrc (refresh your currect session source ~/.bashrc)
Finally, now that you have all the tools you need, proceed to creating a new project
-
run the android plugin project setup tool: ~/bin/g8 jberkel/android-app, follow the onscreen questions (or press return for defaults) this will create the project folder and create the files needed to build an android project
My take on: How to give your new project a good starting point
So, you have a new shiny project on your lap, so now what? Today everyone is talking about “agile”, and how its better then the old “waterfall” methodology of doing things
-
Waterfall
The “Waterfall” approach is notorious in the software community, some highlights of why this is the case:
-
-
It assumes that the initial requirements will not change
-
There is only one version delivered to the stakeholders, at the end of the process
-
All the planning and “thinking” happen only in the beginning of the project
-
-
Agile
Everyone likes “Agile”, its the “cool” way to manage a project, it asks for less paperwork, less work upfront, built from the ground up to support changes, to name a few conceptions about it. While I call it “Agile” here, in reality it is a world of sub cultures, some of which are: “XP” (eXtreme Programming), “scrum”, “Kanban” to name a few. The difference between specific cultures is not always clear, we don’t necessarily have to stick to one, we can mix and match, like I am doing here, thus from this point onward I will just ignorantly call it “Agile” with out specifying the specific methodology. The main strengths going for the Agile methodology are
-
-
Short upfront work phase (requirements, design ..)
-
Iterations - project is broken down to deliverables, which is translated to real value delivered very early in the project lifetime
-
Changes are welcome and expected and are introduced into following iterations (bugs found in previous deliverables, design “inadequacies”)
-
-
Doing things in small pieces is great, just ask a WEB developer, how there’s nothing quite like writing a bit of code, refreshing the page and seeing the changes reflected, especially when working on a big complex project. This gives you a feeling of progress and achievement, even though the end goal is far far away..
-
Some Agile approaches talk beyond writing business code and going into proactive bug searching using TDD (Test Driven Development), which turns out to be a good fit to the overall process as it secures quality of delivered pieces of logic. BDD (Behavior Driven Development) is very similar to TDD, but instead of going after “test coverage” it takes on a higher level, business specification driven approach, where the business embodied user stories are part of the actual test suite.
-
My recipe for success
-
User Stories
-
Mock-ups (in the case of an app that has UI)
-
HLD (High Level -system component- Design)
-
Process Sequence diagram / Workflow diagram (possibly atop HLD)
-
-
Tooling
Tools that keep me productive:
-
-
Google Document (User Stories)
-
Google Drawing (mockups, diagrams, HLD)
-
Lucidchart (diagrams)
-
-
In addition, I use a light weight web based project management tool as a central place to keep reference to important docs and user stories, and see my progress (and others working on the project). A great tool for the job is Trello.com - which free and super simple!
System Architect - mostly interested in the software stack, deployment and the database design
-
Team Members - mostly interested in the more lower level stuff like modules and API’s
-
Other Teams - mostly interested in API’s
-
-
Feedback on design is great for everyone involved, setup meetings with relevant parties to do DR (design review)
-
UML and Other Vegetables
UML is complex, you don’t need to know it perfectly, start with what makes sense, and slowly pick up tutorials. Remember, its a visual tool that helps you communicate, not everyone who will look at your designs has the time (or cares for) to read throughly a book about UML and its specifics. In Einstein’s words “Make everything as simple as possible, but not simpler.”
-
Disclosure
Design does not last. Nobody is going to update the design as the system grows and evolves. Having said that, can still be used a good reference.
“Strong typing is for people with weak memories” - not sure who originally said that, but you have got to love the enthusiasm people have for dynamic languges. What is there not to love? powerful basic language constructs such as closure, prototypal inheritance, objects as first class citizens and anonymous functions, mainly, with which you can simulate almost anything static languages have to offer!
-
Proof you say? no problem! here are few implementations that are worthy of noting:
(if you have other interesting examples let me know!)
-
Convention over configuration
The combination of CoffeeScript’s “classical” Class like system and Backbone.JS’s MVC style structure, give (IMHO) the best value in terms of complexity and pure fun writing apps. The challenges I come across while building large complex applications is the inherited nature of a true dynamic language “typelessness“.
-
The ugly monster, that is -a large application written in JavaScript, pushed me into struggling with these two main challenges:
-
-
Many-to-many PubSub communication
-
3rd party API’s
-
-
Many-to-many PubSub communication
While the PubSub (also known as observer pattern) in JavaScript is a powerful tool to decouple your code, having multiple callers and listeners on the same event makes it really difficult to keep track of the actual common API for these events. Lats try to reflect on this while looking at an actual implementation.
-
(please excuse my CoffeeScript)
-
Pub/Sub implementation in Backbone/Underscore.JS
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14
pubsub = _.extend {}, Backbone.Events pubsub.on 'myEvent', (param1, param2)-> x1 = param1 + param2 # do something with x1
# in some other obscure location in the code pubsub.on 'myEvent', (param1)-> x2 = param1 # do something with x2
The “pubsub.on(event name, callback)” API is very common amongst different JS pub/sub frameworks. here’s an example of how a generic implementation might look like:
If your using the very popular Socket.IO library, for realtime client/server communication, you are using the exact same API, in which case it makes it even more complicated to keep track of, since the ambiguity is manifested on both the client and the server.
-
3rd party API’s
This is how a type related bug might look like in a dynamic language
-
1 2 3
a = "42" # returned from some API a == 42 => false
I’m introducing a convention here, that helps keep the API consistent across the application, the approach here is to create wrapper functions, that are used as both the API documentation and run-time validation facility for required and optional arguments.
-
Example:
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
# declare the events and their APIs Messages = { LoginEvent : Contractor.Create( "LoginEvent", Contractor.Required("user ID"), Contractor.Optional("additional info") ) AppErrorEvent : Contractor.Create( "AppErrorEvent", Contractor.Required("user ID"), Contractor.Required("error description"), Contractor.Optional("exception") ) }
# register on event API pubsub.on Messages.LoginEvent, (userID, info)-> #... do stuff here pubsub.on Messages.AppErrorEvent, (userID, description, exception)-> #... do stuff here
# trigger the event pubsub.apply(pubsub, Messages.LoginEvent(112233, "mobile")) pubsub.apply(pubsub, Messages.LoginEvent(112233))
pubsub.apply(pubsub, Messages.AppErrorEvent(112233, "user did not have access to resource", e)) pubsub.apply(pubsub, Messages.AppErrorEvent(112233)) # since we did not provide the second required argument, this will log an error and return null
-
With NodeJS, I am using the same code on the front and back end, which makes it really easy to manage my Socket.IO communication.
-
If you have been paying attention, there’s still the type issue. Since I work allot with Backbone, it made a lot of sense to write “backbone-typed” which adds optional typing to Backbone-Models. Example:
Unlike the classic FIFO queue, jobs are scheduled for execution in the future, possibly on a specific day, week, month year.. similar to “cron” in a Unix OS. Unlike the normal Unix “cron” implementation, we want it to be distributed, so that several machines(workers) can query and then execute these jobs.
-
Implementing
-
Planning - The standart Unix cron format is very flexible and has been time tested for describing schedules
-
Managing/Coordinating - The queues are stored in Redis Sorted Sets, Lua will be used for atomic execution
-
Execution - Getting the pieces together
-
-
Planning
“Unix Time” format, is the number of milliseconds since (UTC) 1 January 1970 Example (* the actual unix time number is in UTC, while the formatted date is localized)
-
1350745504313 == Sat Oct 20 2012 17:05:04 GMT+0200 (IST)
-
Using this format will make it possible to slice the queue with specific point in time in mind.
-
Managing/Coordinating
Redis’st will be holding the job IDs as members scored by their corresponding execution times(in unix time format).
Finding a job to run: ZRANGEBYSCORE cron:queue 1350745504314 LIMIT 0 1
-
-
Execution
Without going into any platform specific implementation details, this is what the process looks like
-
-
When a new schedule is added, jobs are created for upcoming execution times, which are calculated for the next two hours and added to our cron queue
-
A special job is added to be run every hour, it will run over all the schedules, calculate and add their execution time to the scheduled queue, two hours into the future
-
Workers will be polling the database every second to find jobs
-
Once an entry comes up, it is removed from the queue and added to another Sorted Set queue, so we dont “forget” any jobs..
-
-
As for items 3 and 4 above, it is not possible to implement this logic in the client, because for example when we find a job that needs to be executed, another worker might grab it, or worse they both “grabbed” it and are now working on the same job. To work around this, we need to write a little Lua script, since Lua scripts are executed serially, just like any other Redis command
local currentTimeMS = KEYS[1] local workingQueue = "scheduler:working" local scheduleQueue = "schedule" -- for easier reading, epoch time is in milliseconds local second = 1000
localfunctionfoundJob(jobID) -- implement removing the job from all other queues -- implement updating job status where the actual job object is stored end
localfunctionfindNextJob() local result -- check if there are any "stale" jobs result = redis.call( 'ZRANGEBYSCORE', workingQueue, 0, currentTimeMS-60*second, 'LIMIT',0,1 ) if result[1] ~= nilthenreturn foundJob(result[1]) end -- check the normal queue result = redis.call( 'ZRANGEBYSCORE', scheduleQueue, 0, currentTimeMS, 'LIMIT',0,1 ) if result[1] ~= nilthenreturn foundJob(result[1]) end -- if we didnt find anything return nil returnnil end
return findNextJob()
-
And there you have it! a “Distributed scheduled queue” implemented on top of Redis.
-
If you are going to implement this with node, I created a little cron format parser you can find here: JSCron
Contractor is a factory of contracts, that helps with documenting our APIs. The child_process facility built into node is using EventEmitter (v1), and does not support callbacks, so it is a great fit for Contractor! Since I needed it for a distributed worker framework I am working on, I added a helper facility called “Lawyer”, it basically reads the contracts and routes them to the provided object.
-
Let us take the parent and child example code from Node’s official documentation (http://nodejs.org/api/child_process.html) and see how simple it would be to add some documentation.
Node’s child_process facility provied us with a very basic means of communication, we simply call n.send({any: "thing"}) and it automagically recieved by the forked child. Let’s spice it up with a more RPC message passing style, with contracts and lawyers
var contracts = require('./contracts'); var Contractor = require('contractor').Contractor; var Lawyer = require('contractor').Lawyer; var cp = require('child_process');
The Kotlin project page has this to say about Kotlin: “is a statically typed programming language that compiles to JVM byte codes and JavaScript.” project page
-
Kotlin is a very interesting project, I wanted to check it out, so I decided to write this blog post exploring some design aspects behind Kotlin, and specifically how it compares to the leading JVM “hacker” language - Scala.
-
Who Is Behind Kotlin?
Even though there are several decent free IDE’s available today (Eclipse, NetBeans, Visual Studio Express..), it (could be) surprising that JetBrains is able to rack in between 199$ for a personal license and up to 699$ for a commercial license (for companies) including 1 year update subscription (I have indeed bought one personal license for my development needs). If you are developing on top of VisualStudio you are (most) probably familiar with the 250$+ plugin ReSharper.
-
I would argue that this makes them a good candidate, with insight into good and bad language features, but lets not get ahead of ourselves and set the expectations too high.
-
Why Another JVM Language?
JVM is a well known platform, its weaknesses and scalability strategies are available and documented well on the web, its being used as classic Java run-time VM and for some other popular languages like Scala, Groovy, Clojure and others. Many modules have been already written in Java, normally JVM run languages are compatible with the existing Java ecosystem.
-
Why Static Typing?
In the old (and still ongoing) battle of dynamic and static languages, the consensus is that dynamic languages are faster and easier way to get things done, on the static languages side they allow programmers harness the power of intelligent IDEs to scale up and maintain their code. Type-inferred languages try to find a middle ground approach so that you don’t need to write so much boilerplate code (cue JAVA) to support the static typing, while having the benefits of a full compile time type enforcement.
-
Finally, Kotlin Vs. Scala - Fight!
Scala today is the fastest growing JVM language and is probably competing for the same crowd as Kotlin, Kotlin put up a comparison page link so we will simply go over the omissions part and try to make sense of it.
-
OMITTED - Implicit conversions, parameters
What is it? Scala implicits is (IMHO) a single biggest weakness in the language, once they are used, along with the ability to boast symbols for syntax, it quickly stops being human readable and more like a DSL. Which requires many trips to the docs or even worse - the source code.
-
SBT (Scala Build Tool) is the Maven/Ant of the Scala world link, its very difficult to get things done with it without many trips to google/documentation
Lift Is a popular web framework, here’s a snippet I took from one of its documentation pages link (see the "#result *" #> x part)
-
1 2 3 4 5 6 7 8 9
defrender= inputParam match { caseFull(x) => println("Input is: "+x) "#result *" #> x case _ => println("No input present! Rendering input form HTML") PassThru }
-
Thoughts - Indeed for many people this is the part that makes Kotlin a better “scalable” language, by not allowing programmers and libraries hinder the code unreadable without an IDE
-
OMITTED - Overridable Type Members
What is it? Type members in Scala allow you to manage local generic types by assign a type to a class/abstract class/trait member
-
1 2 3
abstractclassHasTypedMember{ typetypeMe }
-
An extending class will provide a specific type for this type type member
-
1 2 3
classExtendingClassextendsHasTypedMember{ typetypeMe= Int }
-
Thoughts - Frankly, its not really clear why Kotlin decided to omit having typed members, I could be missing something.. Never the less Kotlin usually takes the minimalistic and simplistic route, my guess is that if this is indeed useful, the door is open to add it later..
-
OMITTED - Path-dependent types
What is it? Pointing to a class under another class in Java would mean a specific type (Parent.Child), well not so in Scala. Scala makes the nested class of two same class instances different, much like you would expect two members of two different instances of a single class. Confused? here is some code to explain this better
-
Java:
-
1 2 3 4 5 6 7 8 9 10 11 12
publicclassOuter{ publicclassInner{} publicvoiduseInner(Outer.Inner inner){...} } //.. somewhere in the code o1 = new Outer() o2 = new Outer() i1 = new o1.inner() i2 = new o2.inner() // this would work fine o1.useInner(i2) // works o2.useInner(i1) // works
-
Scala:
1 2 3 4 5 6 7 8 9 10 11 12
classOuter{ classInner{} defuseInner(inner : Inner){...} } //.. somewhere in the code.. val o1 = newOuter val o2 = newOuter val i1 = new o1.Inner val i2 = new o2.Inner o1.useInner(i1) // works fine o2.useInner(i2) // works fine o1.useInner(i2) // THROWS type mismatch exception!
-
Thoughts - It makes some sense that the path to a type that can only exist in an instance is enforced as such.. It would be interesting to know why Kotlin decided to implement this feature in a similar way to Java (conservative?)
-
OMITTED - Existential types
What is it? This is yet another trade-off for Scala to be able to inter-op with Java Let’s see an example:
-
1 2 3 4 5
// Create a case class with generic type caseclassZ[T]( blah:T) // create an instance and cast it to a type created with existential type annotation Seq( Z[Int]( 5 ), Z[String]("blah") ) : Seq[Z[X] forSome { typeX>: StringwithInt }] res1 = Seq[Z[_ >: StringwithInt]] = List(Z(5), Z(blah))
-
Thoughts - Scala supports type variance to describe generic type bounds, Scala designer (Odersky) chose to add existential types support due to three main reasons link: erasure, raw types (lack of generics) and Java’s “wildcard” types.
-
Kotlin took the more “purist” approach (also known as reified types) of dropping support for erasure, raw types, wildcard types and finally existential types.
-
OMITTED - Complicated logic for initialization of traits
What is it? Traits are similar to abstract classes, except that you may use several traits on the same class at the same time (like interfaces).
-
In Scala, traits are constructed before the extended classes
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14
traitAverager{ val first : Int val second : Int val avg : Int = first / second } // if we were to naively construct a class like so: classNaiveextendsAverager{ val first = 20 val second = 10 } // create instance > newNaive() java.lang.ArithmeticException: / by zero // what happened was that our val "avg" was computed at construction time before seeing the values we put in Naive
-
Alternatively the trait can be used with a pre-initialized anonymous class val res = new { val first = 20 ; val second = 10 } with Averager, also the val which depends on other vals (avg in our case) can be made “lazy” with the (obvious) lazy keyword lazy val avg : Int = first / second.
-
As a side note, using def for “avg” could work, but it would mean evaluating whatever is defined under “avg” every-time this method is called.
-
Thoughts - Traits in Scala can have state which could be made tricky. If you read this blog post link you will see Kotlin developers thoughts and a nice insight into what went into designing the inheritance in Kotlin. (don’t forget to read the comments!)
-
OMITTED - Custom symbolic operations
What is it? Scala does not have a built in handling of symbols (like =,+,/ etc), they are actually method calls, as you can have symbols used for method names.
-
Thoughts - This may sound useful at first, but can and (arguably) will create very confusing looking code, like we saw before in the Lift and SBT examples.
-
OMITTED - Built-in XML
What is it? Exactly as it sounds, Scala built in XML native support, so you can do stuff like this
-
1
val holdsXml = <outer><inner>some text</inner></outer>
-
Thoughts - Having a built in support for XML as a language feature is interesting, and sometimes useful. An interview link with Odersky reveals that he would consider removing the native XML support in Scala if had written Scala now.
-
Bottom Line
Kotlin is an idiomatic language that draws from the many good features in todays leading languages, specifically from Scala which is a leading language on the JVM platform today. We’ve looked into omissions on Kotlin’s part versus Scala
-
I’ve very much enjoyed writing this blog post and learning about the evolution that the design of Java/Scala/Kotlin languages, and the choices the respective authors made along the way.
-
To get a better feeling of Kotlin I might pick it up for a small project, at which point I will surely blog about it some more..
The Kotlin project page has this to say about Kotlin: “is a statically typed programming language that compiles to JVM byte codes and JavaScript.” project page
-
Kotlin is a very interesting project, I wanted to check it out, so I decided to write this blog post exploring some design aspects behind Kotlin, and specifically how it compares to the leading JVM “hacker” language - Scala.
-
Who Is Behind Kotlin?
Even though there are several decent free IDE’s available today (Eclipse, NetBeans, Visual Studio Express..), it (could be) surprising that JetBrains is able to rack in between 199$ for a personal license and up to 699$ for a commercial license (for companies) including 1 year update subscription (I have indeed bought one personal license for my development needs). If you are developing on top of VisualStudio you are (most) probably familiar with the 250$+ plugin ReSharper.
-
I would argue that this makes them a good candidate, with insight into good and bad language features, but lets not get ahead of ourselves and set the expectations too high.
-
Why Another JVM Language?
JVM is a well known platform, its weaknesses and scalability strategies are available and documented well on the web, its being used as classic Java run-time VM and for some other popular languages like Scala, Groovy, Clojure and others. Many modules have been already written in Java, normally JVM run languages are compatible with the existing Java ecosystem.
-
Why Static Typing?
In the old (and still ongoing) battle of dynamic and static languages, the consensus is that dynamic languages are faster and easier way to get things done, on the static languages side they allow programmers harness the power of intelligent IDEs to scale up and maintain their code. Type-inferred languages try to find a middle ground approach so that you don’t need to write so much boilerplate code (cue JAVA) to support the static typing, while having the benefits of a full compile time type enforcement.
-
Finally, Kotlin Vs. Scala - Fight!
Scala today is the fastest growing JVM language and is probably competing for the same crowd as Kotlin, Kotlin put up a comparison page link so we will simply go over the omissions part and try to make sense of it.
-
OMITTED - Implicit conversions, parameters
What is it? Scala implicits is (IMHO) a single biggest weakness in the language, once they are used, along with the ability to boast symbols for syntax, it quickly stops being human readable and more like a DSL. Which requires many trips to the docs or even worse - the source code.
-
SBT (Scala Build Tool) is the Maven/Ant of the Scala world link, its very difficult to get things done with it without many trips to google/documentation
Lift Is a popular web framework, here’s a snippet I took from one of its documentation pages link (see the "#result *" #> x part)
-
1 2 3 4 5 6 7 8 9
defrender= inputParam match { caseFull(x) => println("Input is: "+x) "#result *" #> x case _ => println("No input present! Rendering input form HTML") PassThru }
-
Thoughts - Indeed for many people this is the part that makes Kotlin a better “scalable” language, by not allowing programmers and libraries hinder the code unreadable without an IDE
-
OMITTED - Overridable Type Members
What is it? Type members in Scala allow you to manage local generic types by assign a type to a class/abstract class/trait member
-
1 2 3
abstractclassHasTypedMember{ typetypeMe }
-
An extending class will provide a specific type for this type type member
-
1 2 3
classExtendingClassextendsHasTypedMember{ typetypeMe= Int }
-
Thoughts - Frankly, its not really clear why Kotlin decided to omit having typed members, I could be missing something.. Never the less Kotlin usually takes the minimalistic and simplistic route, my guess is that if this is indeed useful, the door is open to add it later..
-
OMITTED - Path-dependent types
What is it? Pointing to a class under another class in Java would mean a specific type (Parent.Child), well not so in Scala. Scala makes the nested class of two same class instances different, much like you would expect two members of two different instances of a single class. Confused? here is some code to explain this better
-
Java:
-
1 2 3 4 5 6 7 8 9 10 11 12
publicclassOuter{ publicclassInner{} publicvoiduseInner(Outer.Inner inner){...} } //.. somewhere in the code o1 = new Outer() o2 = new Outer() i1 = new o1.inner() i2 = new o2.inner() // this would work fine o1.useInner(i2) // works o2.useInner(i1) // works
-
Scala:
1 2 3 4 5 6 7 8 9 10 11 12
classOuter{ classInner{} defuseInner(inner : Inner){...} } //.. somewhere in the code.. val o1 = newOuter val o2 = newOuter val i1 = new o1.Inner val i2 = new o2.Inner o1.useInner(i1) // works fine o2.useInner(i2) // works fine o1.useInner(i2) // THROWS type mismatch exception!
-
Thoughts - It makes some sense that the path to a type that can only exist in an instance is enforced as such.. It would be interesting to know why Kotlin decided to implement this feature in a similar way to Java (conservative?)
-
OMITTED - Existential types
What is it? This is yet another trade-off for Scala to be able to inter-op with Java Let’s see an example:
-
1 2 3 4 5
// Create a case class with generic type caseclassZ[T]( blah:T) // create an instance and cast it to a type created with existential type annotation Seq( Z[Int]( 5 ), Z[String]("blah") ) : Seq[Z[X] forSome { typeX>: StringwithInt }] res1 = Seq[Z[_ >: StringwithInt]] = List(Z(5), Z(blah))
-
Thoughts - Scala supports type variance to describe generic type bounds, Scala designer (Odersky) chose to add existential types support due to three main reasons link: erasure, raw types (lack of generics) and Java’s “wildcard” types.
-
Kotlin took the more “purist” approach (also known as reified types) of dropping support for erasure, raw types, wildcard types and finally existential types.
-
OMITTED - Complicated logic for initialization of traits
What is it? Traits are similar to abstract classes, except that you may use several traits on the same class at the same time (like interfaces).
-
In Scala, traits are constructed before the extended classes
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14
traitAverager{ val first : Int val second : Int val avg : Int = first / second } // if we were to naively construct a class like so: classNaiveextendsAverager{ val first = 20 val second = 10 } // create instance > newNaive() java.lang.ArithmeticException: / by zero // what happened was that our val "avg" was computed at construction time before seeing the values we put in Naive
-
Alternatively the trait can be used with a pre-initialized anonymous class val res = new { val first = 20 ; val second = 10 } with Averager, also the val which depends on other vals (avg in our case) can be made “lazy” with the (obvious) lazy keyword lazy val avg : Int = first / second.
-
As a side note, using def for “avg” could work, but it would mean evaluating whatever is defined under “avg” every-time this method is called.
-
Thoughts - Traits in Scala can have state which could be made tricky. If you read this blog post link you will see Kotlin developers thoughts and a nice insight into what went into designing the inheritance in Kotlin. (don’t forget to read the comments!)
-
OMITTED - Custom symbolic operations
What is it? Scala does not have a built in handling of symbols (like =,+,/ etc), they are actually method calls, as you can have symbols used for method names.
-
Thoughts - This may sound useful at first, but can and (arguably) will create very confusing looking code, like we saw before in the Lift and SBT examples.
-
OMITTED - Built-in XML
What is it? Exactly as it sounds, Scala built in XML native support, so you can do stuff like this
-
1
val holdsXml = <outer><inner>some text</inner></outer>
-
Thoughts - Having a built in support for XML as a language feature is interesting, and sometimes useful. An interview link with Odersky reveals that he would consider removing the native XML support in Scala if had written Scala now.
-
Bottom Line
Kotlin is an idiomatic language that draws from the many good features in todays leading languages, specifically from Scala which is a leading language on the JVM platform today. We’ve looked into omissions on Kotlin’s part versus Scala
-
I’ve very much enjoyed writing this blog post and learning about the evolution that the design of Java/Scala/Kotlin languages, and the choices the respective authors made along the way.
-
To get a better feeling of Kotlin I might pick it up for a small project, at which point I will surely blog about it some more..
Unlike the classic FIFO queue, jobs are scheduled for execution in the future, possibly on a specific day, week, month year.. similar to “cron” in a Unix OS. Unlike the normal Unix “cron” implementation, we want it to be distributed, so that several machines(workers) can query and then execute these jobs.
-
Implementing
-
Planning - The standart Unix cron format is very flexible and has been time tested for describing schedules
-
Managing/Coordinating - The queues are stored in Redis Sorted Sets, Lua will be used for atomic execution
-
Execution - Getting the pieces together
-
-
Planning
“Unix Time” format, is the number of milliseconds since (UTC) 1 January 1970 Example (* the actual unix time number is in UTC, while the formatted date is localized)
-
1350745504313 == Sat Oct 20 2012 17:05:04 GMT+0200 (IST)
-
Using this format will make it possible to slice the queue with specific point in time in mind.
-
Managing/Coordinating
Redis’st will be holding the job IDs as members scored by their corresponding execution times(in unix time format).
Finding a job to run: ZRANGEBYSCORE cron:queue 1350745504314 LIMIT 0 1
-
-
Execution
Without going into any platform specific implementation details, this is what the process looks like
-
-
When a new schedule is added, jobs are created for upcoming execution times, which are calculated for the next two hours and added to our cron queue
-
A special job is added to be run every hour, it will run over all the schedules, calculate and add their execution time to the scheduled queue, two hours into the future
-
Workers will be polling the database every second to find jobs
-
Once an entry comes up, it is removed from the queue and added to another Sorted Set queue, so we dont “forget” any jobs..
-
-
As for items 3 and 4 above, it is not possible to implement this logic in the client, because for example when we find a job that needs to be executed, another worker might grab it, or worse they both “grabbed” it and are now working on the same job. To work around this, we need to write a little Lua script, since Lua scripts are executed serially, just like any other Redis command
local currentTimeMS = KEYS[1] local workingQueue = "scheduler:working" local scheduleQueue = "schedule" -- for easier reading, epoch time is in milliseconds local second = 1000
localfunctionfoundJob(jobID) -- implement removing the job from all other queues -- implement updating job status where the actual job object is stored end
localfunctionfindNextJob() local result -- check if there are any "stale" jobs result = redis.call( 'ZRANGEBYSCORE', workingQueue, 0, currentTimeMS-60*second, 'LIMIT',0,1 ) if result[1] ~= nilthenreturn foundJob(result[1]) end -- check the normal queue result = redis.call( 'ZRANGEBYSCORE', scheduleQueue, 0, currentTimeMS, 'LIMIT',0,1 ) if result[1] ~= nilthenreturn foundJob(result[1]) end -- if we didnt find anything return nil returnnil end
return findNextJob()
-
And there you have it! a “Distributed scheduled queue” implemented on top of Redis.
-
If you are going to implement this with node, I created a little cron format parser you can find here: JSCron
Contractor is a factory of contracts, that helps with documenting our APIs. The child_process facility built into node is using EventEmitter (v1), and does not support callbacks, so it is a great fit for Contractor! Since I needed it for a distributed worker framework I am working on, I added a helper facility called “Lawyer”, it basically reads the contracts and routes them to the provided object.
-
Let us take the parent and child example code from Node’s official documentation (http://nodejs.org/api/child_process.html) and see how simple it would be to add some documentation.
Node’s child_process facility provied us with a very basic means of communication, we simply call n.send({any: "thing"}) and it automagically recieved by the forked child. Let’s spice it up with a more RPC message passing style, with contracts and lawyers
var contracts = require('./contracts'); var Contractor = require('contractor').Contractor; var Lawyer = require('contractor').Lawyer; var cp = require('child_process');
“Strong typing is for people with weak memories” - not sure who originally said that, but you have got to love the enthusiasm people have for dynamic languges. What is there not to love? powerful basic language constructs such as closure, prototypal inheritance, objects as first class citizens and anonymous functions, mainly, with which you can simulate almost anything static languages have to offer!
-
Proof you say? no problem! here are few implementations that are worthy of noting:
(if you have other interesting examples let me know!)
-
Convention over configuration
The combination of CoffeeScript’s “classical” Class like system and Backbone.JS’s MVC style structure, give (IMHO) the best value in terms of complexity and pure fun writing apps. The challenges I come across while building large complex applications is the inherited nature of a true dynamic language “typelessness“.
-
The ugly monster, that is -a large application written in JavaScript, pushed me into struggling with these two main challenges:
-
-
Many-to-many PubSub communication
-
3rd party API’s
-
-
Many-to-many PubSub communication
While the PubSub (also known as observer pattern) in JavaScript is a powerful tool to decouple your code, having multiple callers and listeners on the same event makes it really difficult to keep track of the actual common API for these events. Lats try to reflect on this while looking at an actual implementation.
-
(please excuse my CoffeeScript)
-
Pub/Sub implementation in Backbone/Underscore.JS
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14
pubsub = _.extend {}, Backbone.Events pubsub.on 'myEvent', (param1, param2)-> x1 = param1 + param2 # do something with x1
# in some other obscure location in the code pubsub.on 'myEvent', (param1)-> x2 = param1 # do something with x2
The “pubsub.on(event name, callback)” API is very common amongst different JS pub/sub frameworks. here’s an example of how a generic implementation might look like:
If your using the very popular Socket.IO library, for realtime client/server communication, you are using the exact same API, in which case it makes it even more complicated to keep track of, since the ambiguity is manifested on both the client and the server.
-
3rd party API’s
This is how a type related bug might look like in a dynamic language
-
1 2 3
a = "42" # returned from some API a == 42 => false
I’m introducing a convention here, that helps keep the API consistent across the application, the approach here is to create wrapper functions, that are used as both the API documentation and run-time validation facility for required and optional arguments.
-
Example:
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
# declare the events and their APIs Messages = { LoginEvent : Contractor.Create( "LoginEvent", Contractor.Required("user ID"), Contractor.Optional("additional info") ) AppErrorEvent : Contractor.Create( "AppErrorEvent", Contractor.Required("user ID"), Contractor.Required("error description"), Contractor.Optional("exception") ) }
# register on event API pubsub.on Messages.LoginEvent, (userID, info)-> #... do stuff here pubsub.on Messages.AppErrorEvent, (userID, description, exception)-> #... do stuff here
# trigger the event pubsub.apply(pubsub, Messages.LoginEvent(112233, "mobile")) pubsub.apply(pubsub, Messages.LoginEvent(112233))
pubsub.apply(pubsub, Messages.AppErrorEvent(112233, "user did not have access to resource", e)) pubsub.apply(pubsub, Messages.AppErrorEvent(112233)) # since we did not provide the second required argument, this will log an error and return null
-
With NodeJS, I am using the same code on the front and back end, which makes it really easy to manage my Socket.IO communication.
-
If you have been paying attention, there’s still the type issue. Since I work allot with Backbone, it made a lot of sense to write “backbone-typed” which adds optional typing to Backbone-Models. Example:
System Architect - mostly interested in the software stack, deployment and the database design
-
Team Members - mostly interested in the more lower level stuff like modules and API’s
-
Other Teams - mostly interested in API’s
-
-
Feedback on design is great for everyone involved, setup meetings with relevant parties to do DR (design review)
-
UML and Other Vegetables
UML is complex, you don’t need to know it perfectly, start with what makes sense, and slowly pick up tutorials. Remember, its a visual tool that helps you communicate, not everyone who will look at your designs has the time (or cares for) to read throughly a book about UML and its specifics. In Einstein’s words “Make everything as simple as possible, but not simpler.”
-
Disclosure
Design does not last. Nobody is going to update the design as the system grows and evolves. Having said that, can still be used a good reference.
“User Story” is an important communication tool, it is used between you (the developer) and who ever is making the requirements (the stakeholder). A “contract” which (unlike one written by a lawyer) is clear, understood and agreed upon. On our side (development) user stories embody a work unit that can be further broken down and estimated.
-
Key value points:
-
-
Language - its key that user stories are written in a language everyone can understand (stakeholders, developers etc..)
-
Perspective - user stories need to be written from the perspective of the “feature user”, so they represent real world value
-
Estimation - to help you get better and more precise estimation for the project overall
-
Prioritization - what needs to be delivered soon/later
-
-
Lets imagine we just got a call from the product manager, where he is telling us about this new survey tool:
-
It needs to collect answers from users via a link sent to our users by e-mail
-
It needs to be summarized and sent to … once a month on the 1st
-
Managers need to be able to see past surveys
-
-
Hmm.. OK, thats pretty clear, but suppose this can be written clearer, lats try using the following template: (not my invention;)) As a [role], I want [Requirement / Feature], So that [Justigication/ Reason]
-
The requirements now look like this:
-
-
As an end user, I will be able to follow a link I got via e-mail, so that I can fill a survey and express my opinion
-
As a manager, I will receive an e-mail with a digest of surveys, so that I can evaluate performance
-
As a manager, I will be able to access old surveys, so that I can compare and evaluate performance over time
-
-
By reading the above I can clearly understand the required functionality and the motivation behind writing it, this gives me a good starting point. Since I am the one to implement this, I will need to provide estimates, so that product can plan ahead and communicate the product road map onward. Unfortunately, “The Devil is In The Details” as they say, so giving estimates based on points 1,2,3 is still difficult. Points 2 & 3 are “huge” stories, also known as “epics”, we need to further break down these stories, while still using the same language and practices:
-
-
As an end user, I will receive an email with a link as a result of some communication with a company, so that I can follow it and fill a survey
-
As an end user, I get a set of questions to answer, so that my opinion is collected
-
As a manager, I will receive an e-mail with a digest of surveys, so that I can evaluate companies performance
-
As a manager, I will have a survey management portal, so that I can overview previous results and compare to
-
As a manager, I can select a specific survey, so that I can extract meaningful actionable information
-
-
The functionality expected of deliverables from the points above is clear, and now I feel more comfortable estimating the time it will take me to complete each. Along with the estimation, these can now be prioritized against each other and against other ongoing projects..
-
Summery
The beginning of a new project is the perfect time to introduce user stories, it will help with getting the conversation going, further clarifying the scope and requirements. While the project scope might change, user stories are (optimally) modular, decoupled and encapsulated, this should help with managing changes and their effect on the overall project.
from the list select the Android version your afer, right now its Android 4.0.3 (API 15) (I also installed Android 2.3.3), click on Install packages to get the installation started
-
-
setup your env echo "export ANDROID_HOME=~/Applications/android-sdk-linux/" >> ~/.bashrc
-
-
refresh your currect session source ~/.bashrc
-
-
-
Adding SBT support to IntelliJ
-
open File > Settings > Plugin
-
-
click on Browser repositories
-
-
find SBT, right click and select Download and install
create folder ~/bin/ (if you dont have ~/bin in your PATH, then you can do this: echo "PATH=$PATH:~/bin" >> ~/.bashrc (refresh your currect session source ~/.bashrc)
Finally, now that you have all the tools you need, proceed to creating a new project
-
run the android plugin project setup tool: ~/bin/g8 jberkel/android-app, follow the onscreen questions (or press return for defaults) this will create the project folder and create the files needed to build an android project
My take on: How to give your new project a good starting point
So, you have a new shiny project on your lap, so now what? Today everyone is talking about “agile”, and how its better then the old “waterfall” methodology of doing things
-
Waterfall
The “Waterfall” approach is notorious in the software community, some highlights of why this is the case:
-
-
It assumes that the initial requirements will not change
-
There is only one version delivered to the stakeholders, at the end of the process
-
All the planning and “thinking” happen only in the beginning of the project
-
-
Agile
Everyone likes “Agile”, its the “cool” way to manage a project, it asks for less paperwork, less work upfront, built from the ground up to support changes, to name a few conceptions about it. While I call it “Agile” here, in reality it is a world of sub cultures, some of which are: “XP” (eXtreme Programming), “scrum”, “Kanban” to name a few. The difference between specific cultures is not always clear, we don’t necessarily have to stick to one, we can mix and match, like I am doing here, thus from this point onward I will just ignorantly call it “Agile” with out specifying the specific methodology. The main strengths going for the Agile methodology are
-
-
Short upfront work phase (requirements, design ..)
-
Iterations - project is broken down to deliverables, which is translated to real value delivered very early in the project lifetime
-
Changes are welcome and expected and are introduced into following iterations (bugs found in previous deliverables, design “inadequacies”)
-
-
Doing things in small pieces is great, just ask a WEB developer, how there’s nothing quite like writing a bit of code, refreshing the page and seeing the changes reflected, especially when working on a big complex project. This gives you a feeling of progress and achievement, even though the end goal is far far away..
-
Some Agile approaches talk beyond writing business code and going into proactive bug searching using TDD (Test Driven Development), which turns out to be a good fit to the overall process as it secures quality of delivered pieces of logic. BDD (Behavior Driven Development) is very similar to TDD, but instead of going after “test coverage” it takes on a higher level, business specification driven approach, where the business embodied user stories are part of the actual test suite.
-
My recipe for success
-
User Stories
-
Mock-ups (in the case of an app that has UI)
-
HLD (High Level -system component- Design)
-
Process Sequence diagram / Workflow diagram (possibly atop HLD)
-
-
Tooling
Tools that keep me productive:
-
-
Google Document (User Stories)
-
Google Drawing (mockups, diagrams, HLD)
-
Lucidchart (diagrams)
-
-
In addition, I use a light weight web based project management tool as a central place to keep reference to important docs and user stories, and see my progress (and others working on the project). A great tool for the job is Trello.com - which free and super simple!
If you need to test/mock 3rd party API’s / a crawler / external web service, and you dont want to use some fancy DI framework (I hate the magic in DI frameworks, I use constructor DI instead..), anyway, here’s a helper class I put togather to that end
-
If you haven’t got a chance to try out Spray, you totally should, its a high performance web server built on top of Netty and Akka
-
Spray has a very nice HTTP API. Few things I liked:
-
-
HTTP method mapping spray.http.HttpMethods - GET, POST etc..
-
HttpRequest and HttpResopnse are basically wrappers around data you would expect to have in a request and response data
-
spray.http.Uri wrapper class around what you would expect to find in the URI
-
it really is just an Akka Actor, with just the right amount of control exposed and the rest well hidden behind its nice APIs
import akka.actor._ import akka.io.IO import java.util.concurrent.atomic.AtomicInteger import spray.can.Http import spray.http.{Uri, HttpMethod, HttpResponse, HttpRequest} objectHelpers{ lazyimplicitval testSystem = ActorSystem("helpers-test-system") // used to track the last assigned port val lastPort = newAtomicInteger(20000) // a single expected triplet of method+uri+body typeReplay= (HttpMethod,Uri.Path,String) // factory method to get a replater to play around with defwebReplayer(replay : List[Replay]) : (ActorRef,Int) = { // everytime the factory is used a new port is assigned, // so many instances can co-exist and tests can run in parallel without getting // on eachother's toes val newPort = lastPort.addAndGet(1) // instantiate a new Spray server. Yes, its that simple! val newServer = testSystem.actorOf(Props(newHelpers.ReplayerActor(replay))) IO(Http) ! Http.Bind(newServer, interface = "127.0.0.1", port = newPort) // return a pair of the server instance and the assigned port // the newPort should be used by client code to construct the remote URI with (newServer,newPort) } // object to use to tell the server to shutdown.. objectShutTestServer // a concrete implementation for a Spray Actor to listen to incoming HttpRequest(..) messages // provided a list of messages to be matched against arrived and content to be sent back classReplayerActor(var replay :List[Replay]) extendsActor{ var ioServer : ActorRef = null overridedefreceive: Receive = { case _ : Http.Connected => sender ! Http.Register(self) ioServer = sender() // this is basically a "catch all" HttpRequest pattern caseHttpRequest(method, uri,_,_,_) => // a test to see that this is indeed expected to happen at this point if (!replay.isEmpty && replay.head._1 == method && uri.path == replay.head._2){ sender ! HttpResponse(entity = replay.head._3) replay = replay.tail } else { sender ! HttpResponse(status = 400, entity = "was expecting a different request at this point") } caseShutTestServer => ioServer ! PoisonPill case _ => //ignore } } }
-
And now a cocrete client code example to use this tool
// some random content I would like to receive in this scenario (or test..) val content = s""" |<html> |<head> |<title>sometitle</title> |</head> |<body>some body</body> |</html> """.stripMargin // we use the factory method to get an instance of "replayer" to play around with val (_testWebServer,port) = Helpers.webReplayer( List( (GET,Uri.Path("/somesite"),content) ) ) // and where ever in your actual code, use the port we got back above and have it "hit" the server val whatEver = YourHTTPGetter.fetch(s"http://127.0.0.1:$port/somesite") assert(whatEver == content)
If you need to test/mock 3rd party API’s / a crawler / external web service, and you dont want to use some fancy DI framework (I hate the magic in DI frameworks, I use constructor DI instead..), anyway, here’s a helper class I put togather to that end
-
If you haven’t got a chance to try out Spray, you totally should, its a high performance web server built on top of Netty and Akka
-
Spray has a very nice HTTP API. Few things I liked:
-
-
HTTP method mapping spray.http.HttpMethods - GET, POST etc..
-
HttpRequest and HttpResopnse are basically wrappers around data you would expect to have in a request and response data
-
spray.http.Uri wrapper class around what you would expect to find in the URI
-
it really is just an Akka Actor, with just the right amount of control exposed and the rest well hidden behind its nice APIs
import akka.actor._ import akka.io.IO import java.util.concurrent.atomic.AtomicInteger import spray.can.Http import spray.http.{Uri, HttpMethod, HttpResponse, HttpRequest} objectHelpers{ lazyimplicitval testSystem = ActorSystem("helpers-test-system") // used to track the last assigned port val lastPort = newAtomicInteger(20000) // a single expected triplet of method+uri+body typeReplay= (HttpMethod,Uri.Path,String) // factory method to get a replater to play around with defwebReplayer(replay : List[Replay]) : (ActorRef,Int) = { // everytime the factory is used a new port is assigned, // so many instances can co-exist and tests can run in parallel without getting // on eachother's toes val newPort = lastPort.addAndGet(1) // instantiate a new Spray server. Yes, its that simple! val newServer = testSystem.actorOf(Props(newHelpers.ReplayerActor(replay))) IO(Http) ! Http.Bind(newServer, interface = "127.0.0.1", port = newPort) // return a pair of the server instance and the assigned port // the newPort should be used by client code to construct the remote URI with (newServer,newPort) } // object to use to tell the server to shutdown.. objectShutTestServer // a concrete implementation for a Spray Actor to listen to incoming HttpRequest(..) messages // provided a list of messages to be matched against arrived and content to be sent back classReplayerActor(var replay :List[Replay]) extendsActor{ var ioServer : ActorRef = null overridedefreceive: Receive = { case _ : Http.Connected => sender ! Http.Register(self) ioServer = sender() // this is basically a "catch all" HttpRequest pattern caseHttpRequest(method, uri,_,_,_) => // a test to see that this is indeed expected to happen at this point if (!replay.isEmpty && replay.head._1 == method && uri.path == replay.head._2){ sender ! HttpResponse(entity = replay.head._3) replay = replay.tail } else { sender ! HttpResponse(status = 400, entity = "was expecting a different request at this point") } caseShutTestServer => ioServer ! PoisonPill case _ => //ignore } } }
-
And now a cocrete client code example to use this tool
// some random content I would like to receive in this scenario (or test..) val content = s""" |<html> |<head> |<title>sometitle</title> |</head> |<body>some body</body> |</html> """.stripMargin // we use the factory method to get an instance of "replayer" to play around with val (_testWebServer,port) = Helpers.webReplayer( List( (GET,Uri.Path("/somesite"),content) ) ) // and where ever in your actual code, use the port we got back above and have it "hit" the server val whatEver = YourHTTPGetter.fetch(s"http://127.0.0.1:$port/somesite") assert(whatEver == content)
If you need to test/mock 3rd party API’s / a crawler / external web service, and you dont want to use some fancy DI framework (I hate the magic in DI frameworks, I use constructor DI instead..), anyway, here’s a helper class I put togather to that end
-
If you haven’t got a chance to try out Spray, you totally should, its a high performance web server built on top of Netty and Akka
-
Spray has a very nice HTTP API. Few things I liked:
-
-
HTTP method mapping spray.http.HttpMethods - GET, POST etc..
-
HttpRequest and HttpResopnse are basically wrappers around data you would expect to have in a request and response data
-
spray.http.Uri wrapper class around what you would expect to find in the URI
-
it really is just an Akka Actor, with just the right amount of control exposed and the rest well hidden behind its nice APIs
import akka.actor._ import akka.io.IO import java.util.concurrent.atomic.AtomicInteger import spray.can.Http import spray.http.{Uri, HttpMethod, HttpResponse, HttpRequest} objectHelpers{ lazyimplicitval testSystem = ActorSystem("helpers-test-system") // used to track the last assigned port val lastPort = newAtomicInteger(20000) // a single expected triplet of method+uri+body typeReplay= (HttpMethod,Uri.Path,String) // factory method to get a replater to play around with defwebReplayer(replay : List[Replay]) : (ActorRef,Int) = { // everytime the factory is used a new port is assigned, // so many instances can co-exist and tests can run in parallel without getting // on eachother's toes val newPort = lastPort.addAndGet(1) // instantiate a new Spray server. Yes, its that simple! val newServer = testSystem.actorOf(Props(newHelpers.ReplayerActor(replay))) IO(Http) ! Http.Bind(newServer, interface = "127.0.0.1", port = newPort) // return a pair of the server instance and the assigned port // the newPort should be used by client code to construct the remote URI with (newServer,newPort) } // object to use to tell the server to shutdown.. objectShutTestServer // a concrete implementation for a Spray Actor to listen to incoming HttpRequest(..) messages // provided a list of messages to be matched against arrived and content to be sent back classReplayerActor(var replay :List[Replay]) extendsActor{ var ioServer : ActorRef = null overridedefreceive: Receive = { case _ : Http.Connected => sender ! Http.Register(self) ioServer = sender() // this is basically a "catch all" HttpRequest pattern caseHttpRequest(method, uri,_,_,_) => // a test to see that this is indeed expected to happen at this point if (!replay.isEmpty && replay.head._1 == method && uri.path == replay.head._2){ sender ! HttpResponse(entity = replay.head._3) replay = replay.tail } else { sender ! HttpResponse(status = 400, entity = "was expecting a different request at this point") } caseShutTestServer => ioServer ! PoisonPill case _ => //ignore } } }
-
And now a cocrete client code example to use this tool
// some random content I would like to receive in this scenario (or test..) val content = s""" |<html> |<head> |<title>sometitle</title> |</head> |<body>some body</body> |</html> """.stripMargin // we use the factory method to get an instance of "replayer" to play around with val (_testWebServer,port) = Helpers.webReplayer( List( (GET,Uri.Path("/somesite"),content) ) ) // and where ever in your actual code, use the port we got back above and have it "hit" the server val whatEver = YourHTTPGetter.fetch(s"http://127.0.0.1:$port/somesite") assert(whatEver == content)
The Kotlin project page has this to say about Kotlin: “is a statically typed programming language that compiles to JVM byte codes and JavaScript.” project page
-
Kotlin is a very interesting project, I wanted to check it out, so I decided to write this blog post exploring some design aspects behind Kotlin, and specifically how it compares to the leading JVM “hacker” language - Scala.
-
Who Is Behind Kotlin?
Even though there are several decent free IDE’s available today (Eclipse, NetBeans, Visual Studio Express..), it (could be) surprising that JetBrains is able to rack in between 199$ for a personal license and up to 699$ for a commercial license (for companies) including 1 year update subscription (I have indeed bought one personal license for my development needs). If you are developing on top of VisualStudio you are (most) probably familiar with the 250$+ plugin ReSharper.
-
I would argue that this makes them a good candidate, with insight into good and bad language features, but lets not get ahead of ourselves and set the expectations too high.
-
Why Another JVM Language?
JVM is a well known platform, its weaknesses and scalability strategies are available and documented well on the web, its being used as classic Java run-time VM and for some other popular languages like Scala, Groovy, Clojure and others. Many modules have been already written in Java, normally JVM run languages are compatible with the existing Java ecosystem.
-
Why Static Typing?
In the old (and still ongoing) battle of dynamic and static languages, the consensus is that dynamic languages are faster and easier way to get things done, on the static languages side they allow programmers harness the power of intelligent IDEs to scale up and maintain their code. Type-inferred languages try to find a middle ground approach so that you don’t need to write so much boilerplate code (cue JAVA) to support the static typing, while having the benefits of a full compile time type enforcement.
-
Finally, Kotlin Vs. Scala - Fight!
Scala today is the fastest growing JVM language and is probably competing for the same crowd as Kotlin, Kotlin put up a comparison page link so we will simply go over the omissions part and try to make sense of it.
-
OMITTED - Implicit conversions, parameters
What is it? Scala implicits is (IMHO) a single biggest weakness in the language, once they are used, along with the ability to boast symbols for syntax, it quickly stops being human readable and more like a DSL. Which requires many trips to the docs or even worse - the source code.
-
SBT (Scala Build Tool) is the Maven/Ant of the Scala world link, its very difficult to get things done with it without many trips to google/documentation
Lift Is a popular web framework, here’s a snippet I took from one of its documentation pages link (see the "#result *" #> x part)
-
1 2 3 4 5 6 7 8 9
defrender= inputParam match { caseFull(x) => println("Input is: "+x) "#result *" #> x case _ => println("No input present! Rendering input form HTML") PassThru }
-
Thoughts - Indeed for many people this is the part that makes Kotlin a better “scalable” language, by not allowing programmers and libraries hinder the code unreadable without an IDE
-
OMITTED - Overridable Type Members
What is it? Type members in Scala allow you to manage local generic types by assign a type to a class/abstract class/trait member
-
1 2 3
abstractclassHasTypedMember{ typetypeMe }
-
An extending class will provide a specific type for this type type member
-
1 2 3
classExtendingClassextendsHasTypedMember{ typetypeMe= Int }
-
Thoughts - Frankly, its not really clear why Kotlin decided to omit having typed members, I could be missing something.. Never the less Kotlin usually takes the minimalistic and simplistic route, my guess is that if this is indeed useful, the door is open to add it later..
-
OMITTED - Path-dependent types
What is it? Pointing to a class under another class in Java would mean a specific type (Parent.Child), well not so in Scala. Scala makes the nested class of two same class instances different, much like you would expect two members of two different instances of a single class. Confused? here is some code to explain this better
-
Java:
-
1 2 3 4 5 6 7 8 9 10 11 12
publicclassOuter{ publicclassInner{} publicvoiduseInner(Outer.Inner inner){...} } //.. somewhere in the code o1 = new Outer() o2 = new Outer() i1 = new o1.inner() i2 = new o2.inner() // this would work fine o1.useInner(i2) // works o2.useInner(i1) // works
-
Scala:
1 2 3 4 5 6 7 8 9 10 11 12
classOuter{ classInner{} defuseInner(inner : Inner){...} } //.. somewhere in the code.. val o1 = newOuter val o2 = newOuter val i1 = new o1.Inner val i2 = new o2.Inner o1.useInner(i1) // works fine o2.useInner(i2) // works fine o1.useInner(i2) // THROWS type mismatch exception!
-
Thoughts - It makes some sense that the path to a type that can only exist in an instance is enforced as such.. It would be interesting to know why Kotlin decided to implement this feature in a similar way to Java (conservative?)
-
OMITTED - Existential types
What is it? This is yet another trade-off for Scala to be able to inter-op with Java Let’s see an example:
-
1 2 3 4 5
// Create a case class with generic type caseclassZ[T]( blah:T) // create an instance and cast it to a type created with existential type annotation Seq( Z[Int]( 5 ), Z[String]("blah") ) : Seq[Z[X] forSome { typeX>: StringwithInt }] res1 = Seq[Z[_ >: StringwithInt]] = List(Z(5), Z(blah))
-
Thoughts - Scala supports type variance to describe generic type bounds, Scala designer (Odersky) chose to add existential types support due to three main reasons link: erasure, raw types (lack of generics) and Java’s “wildcard” types.
-
Kotlin took the more “purist” approach (also known as reified types) of dropping support for erasure, raw types, wildcard types and finally existential types.
-
OMITTED - Complicated logic for initialization of traits
What is it? Traits are similar to abstract classes, except that you may use several traits on the same class at the same time (like interfaces).
-
In Scala, traits are constructed before the extended classes
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14
traitAverager{ val first : Int val second : Int val avg : Int = first / second } // if we were to naively construct a class like so: classNaiveextendsAverager{ val first = 20 val second = 10 } // create instance > newNaive() java.lang.ArithmeticException: / by zero // what happened was that our val "avg" was computed at construction time before seeing the values we put in Naive
-
Alternatively the trait can be used with a pre-initialized anonymous class val res = new { val first = 20 ; val second = 10 } with Averager, also the val which depends on other vals (avg in our case) can be made “lazy” with the (obvious) lazy keyword lazy val avg : Int = first / second.
-
As a side note, using def for “avg” could work, but it would mean evaluating whatever is defined under “avg” every-time this method is called.
-
Thoughts - Traits in Scala can have state which could be made tricky. If you read this blog post link you will see Kotlin developers thoughts and a nice insight into what went into designing the inheritance in Kotlin. (don’t forget to read the comments!)
-
OMITTED - Custom symbolic operations
What is it? Scala does not have a built in handling of symbols (like =,+,/ etc), they are actually method calls, as you can have symbols used for method names.
-
Thoughts - This may sound useful at first, but can and (arguably) will create very confusing looking code, like we saw before in the Lift and SBT examples.
-
OMITTED - Built-in XML
What is it? Exactly as it sounds, Scala built in XML native support, so you can do stuff like this
-
1
val holdsXml = <outer><inner>some text</inner></outer>
-
Thoughts - Having a built in support for XML as a language feature is interesting, and sometimes useful. An interview link with Odersky reveals that he would consider removing the native XML support in Scala if had written Scala now.
-
Bottom Line
Kotlin is an idiomatic language that draws from the many good features in todays leading languages, specifically from Scala which is a leading language on the JVM platform today. We’ve looked into omissions on Kotlin’s part versus Scala
-
I’ve very much enjoyed writing this blog post and learning about the evolution that the design of Java/Scala/Kotlin languages, and the choices the respective authors made along the way.
-
To get a better feeling of Kotlin I might pick it up for a small project, at which point I will surely blog about it some more..
Unlike the classic FIFO queue, jobs are scheduled for execution in the future, possibly on a specific day, week, month year.. similar to “cron” in a Unix OS. Unlike the normal Unix “cron” implementation, we want it to be distributed, so that several machines(workers) can query and then execute these jobs.
-
Implementing
-
Planning - The standart Unix cron format is very flexible and has been time tested for describing schedules
-
Managing/Coordinating - The queues are stored in Redis Sorted Sets, Lua will be used for atomic execution
-
Execution - Getting the pieces together
-
-
Planning
“Unix Time” format, is the number of milliseconds since (UTC) 1 January 1970 Example (* the actual unix time number is in UTC, while the formatted date is localized)
-
1350745504313 == Sat Oct 20 2012 17:05:04 GMT+0200 (IST)
-
Using this format will make it possible to slice the queue with specific point in time in mind.
-
Managing/Coordinating
Redis’st will be holding the job IDs as members scored by their corresponding execution times(in unix time format).
Finding a job to run: ZRANGEBYSCORE cron:queue 1350745504314 LIMIT 0 1
-
-
Execution
Without going into any platform specific implementation details, this is what the process looks like
-
-
When a new schedule is added, jobs are created for upcoming execution times, which are calculated for the next two hours and added to our cron queue
-
A special job is added to be run every hour, it will run over all the schedules, calculate and add their execution time to the scheduled queue, two hours into the future
-
Workers will be polling the database every second to find jobs
-
Once an entry comes up, it is removed from the queue and added to another Sorted Set queue, so we dont “forget” any jobs..
-
-
As for items 3 and 4 above, it is not possible to implement this logic in the client, because for example when we find a job that needs to be executed, another worker might grab it, or worse they both “grabbed” it and are now working on the same job. To work around this, we need to write a little Lua script, since Lua scripts are executed serially, just like any other Redis command
local currentTimeMS = KEYS[1] local workingQueue = "scheduler:working" local scheduleQueue = "schedule" -- for easier reading, epoch time is in milliseconds local second = 1000
localfunctionfoundJob(jobID) -- implement removing the job from all other queues -- implement updating job status where the actual job object is stored end
localfunctionfindNextJob() local result -- check if there are any "stale" jobs result = redis.call( 'ZRANGEBYSCORE', workingQueue, 0, currentTimeMS-60*second, 'LIMIT',0,1 ) if result[1] ~= nilthenreturn foundJob(result[1]) end -- check the normal queue result = redis.call( 'ZRANGEBYSCORE', scheduleQueue, 0, currentTimeMS, 'LIMIT',0,1 ) if result[1] ~= nilthenreturn foundJob(result[1]) end -- if we didnt find anything return nil returnnil end
return findNextJob()
-
And there you have it! a “Distributed scheduled queue” implemented on top of Redis.
-
If you are going to implement this with node, I created a little cron format parser you can find here: JSCron
Contractor is a factory of contracts, that helps with documenting our APIs. The child_process facility built into node is using EventEmitter (v1), and does not support callbacks, so it is a great fit for Contractor! Since I needed it for a distributed worker framework I am working on, I added a helper facility called “Lawyer”, it basically reads the contracts and routes them to the provided object.
-
Let us take the parent and child example code from Node’s official documentation (http://nodejs.org/api/child_process.html) and see how simple it would be to add some documentation.
Node’s child_process facility provied us with a very basic means of communication, we simply call n.send({any: "thing"}) and it automagically recieved by the forked child. Let’s spice it up with a more RPC message passing style, with contracts and lawyers
var contracts = require('./contracts'); var Contractor = require('contractor').Contractor; var Lawyer = require('contractor').Lawyer; var cp = require('child_process');
“Strong typing is for people with weak memories” - not sure who originally said that, but you have got to love the enthusiasm people have for dynamic languges. What is there not to love? powerful basic language constructs such as closure, prototypal inheritance, objects as first class citizens and anonymous functions, mainly, with which you can simulate almost anything static languages have to offer!
-
Proof you say? no problem! here are few implementations that are worthy of noting:
(if you have other interesting examples let me know!)
-
Convention over configuration
The combination of CoffeeScript’s “classical” Class like system and Backbone.JS’s MVC style structure, give (IMHO) the best value in terms of complexity and pure fun writing apps. The challenges I come across while building large complex applications is the inherited nature of a true dynamic language “typelessness“.
-
The ugly monster, that is -a large application written in JavaScript, pushed me into struggling with these two main challenges:
-
-
Many-to-many PubSub communication
-
3rd party API’s
-
-
Many-to-many PubSub communication
While the PubSub (also known as observer pattern) in JavaScript is a powerful tool to decouple your code, having multiple callers and listeners on the same event makes it really difficult to keep track of the actual common API for these events. Lats try to reflect on this while looking at an actual implementation.
-
(please excuse my CoffeeScript)
-
Pub/Sub implementation in Backbone/Underscore.JS
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14
pubsub = _.extend {}, Backbone.Events pubsub.on 'myEvent', (param1, param2)-> x1 = param1 + param2 # do something with x1
# in some other obscure location in the code pubsub.on 'myEvent', (param1)-> x2 = param1 # do something with x2
The “pubsub.on(event name, callback)” API is very common amongst different JS pub/sub frameworks. here’s an example of how a generic implementation might look like:
If your using the very popular Socket.IO library, for realtime client/server communication, you are using the exact same API, in which case it makes it even more complicated to keep track of, since the ambiguity is manifested on both the client and the server.
-
3rd party API’s
This is how a type related bug might look like in a dynamic language
-
1 2 3
a = "42" # returned from some API a == 42 => false
I’m introducing a convention here, that helps keep the API consistent across the application, the approach here is to create wrapper functions, that are used as both the API documentation and run-time validation facility for required and optional arguments.
-
Example:
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
# declare the events and their APIs Messages = { LoginEvent : Contractor.Create( "LoginEvent", Contractor.Required("user ID"), Contractor.Optional("additional info") ) AppErrorEvent : Contractor.Create( "AppErrorEvent", Contractor.Required("user ID"), Contractor.Required("error description"), Contractor.Optional("exception") ) }
# register on event API pubsub.on Messages.LoginEvent, (userID, info)-> #... do stuff here pubsub.on Messages.AppErrorEvent, (userID, description, exception)-> #... do stuff here
# trigger the event pubsub.apply(pubsub, Messages.LoginEvent(112233, "mobile")) pubsub.apply(pubsub, Messages.LoginEvent(112233))
pubsub.apply(pubsub, Messages.AppErrorEvent(112233, "user did not have access to resource", e)) pubsub.apply(pubsub, Messages.AppErrorEvent(112233)) # since we did not provide the second required argument, this will log an error and return null
-
With NodeJS, I am using the same code on the front and back end, which makes it really easy to manage my Socket.IO communication.
-
If you have been paying attention, there’s still the type issue. Since I work allot with Backbone, it made a lot of sense to write “backbone-typed” which adds optional typing to Backbone-Models. Example:
System Architect - mostly interested in the software stack, deployment and the database design
-
Team Members - mostly interested in the more lower level stuff like modules and API’s
-
Other Teams - mostly interested in API’s
-
-
Feedback on design is great for everyone involved, setup meetings with relevant parties to do DR (design review)
-
UML and Other Vegetables
UML is complex, you don’t need to know it perfectly, start with what makes sense, and slowly pick up tutorials. Remember, its a visual tool that helps you communicate, not everyone who will look at your designs has the time (or cares for) to read throughly a book about UML and its specifics. In Einstein’s words “Make everything as simple as possible, but not simpler.”
-
Disclosure
Design does not last. Nobody is going to update the design as the system grows and evolves. Having said that, can still be used a good reference.
“User Story” is an important communication tool, it is used between you (the developer) and who ever is making the requirements (the stakeholder). A “contract” which (unlike one written by a lawyer) is clear, understood and agreed upon. On our side (development) user stories embody a work unit that can be further broken down and estimated.
-
Key value points:
-
-
Language - its key that user stories are written in a language everyone can understand (stakeholders, developers etc..)
-
Perspective - user stories need to be written from the perspective of the “feature user”, so they represent real world value
-
Estimation - to help you get better and more precise estimation for the project overall
-
Prioritization - what needs to be delivered soon/later
-
-
Lets imagine we just got a call from the product manager, where he is telling us about this new survey tool:
-
It needs to collect answers from users via a link sent to our users by e-mail
-
It needs to be summarized and sent to … once a month on the 1st
-
Managers need to be able to see past surveys
-
-
Hmm.. OK, thats pretty clear, but suppose this can be written clearer, lats try using the following template: (not my invention;)) As a [role], I want [Requirement / Feature], So that [Justigication/ Reason]
-
The requirements now look like this:
-
-
As an end user, I will be able to follow a link I got via e-mail, so that I can fill a survey and express my opinion
-
As a manager, I will receive an e-mail with a digest of surveys, so that I can evaluate performance
-
As a manager, I will be able to access old surveys, so that I can compare and evaluate performance over time
-
-
By reading the above I can clearly understand the required functionality and the motivation behind writing it, this gives me a good starting point. Since I am the one to implement this, I will need to provide estimates, so that product can plan ahead and communicate the product road map onward. Unfortunately, “The Devil is In The Details” as they say, so giving estimates based on points 1,2,3 is still difficult. Points 2 & 3 are “huge” stories, also known as “epics”, we need to further break down these stories, while still using the same language and practices:
-
-
As an end user, I will receive an email with a link as a result of some communication with a company, so that I can follow it and fill a survey
-
As an end user, I get a set of questions to answer, so that my opinion is collected
-
As a manager, I will receive an e-mail with a digest of surveys, so that I can evaluate companies performance
-
As a manager, I will have a survey management portal, so that I can overview previous results and compare to
-
As a manager, I can select a specific survey, so that I can extract meaningful actionable information
-
-
The functionality expected of deliverables from the points above is clear, and now I feel more comfortable estimating the time it will take me to complete each. Along with the estimation, these can now be prioritized against each other and against other ongoing projects..
-
Summery
The beginning of a new project is the perfect time to introduce user stories, it will help with getting the conversation going, further clarifying the scope and requirements. While the project scope might change, user stories are (optimally) modular, decoupled and encapsulated, this should help with managing changes and their effect on the overall project.
from the list select the Android version your afer, right now its Android 4.0.3 (API 15) (I also installed Android 2.3.3), click on Install packages to get the installation started
-
-
setup your env echo "export ANDROID_HOME=~/Applications/android-sdk-linux/" >> ~/.bashrc
-
-
refresh your currect session source ~/.bashrc
-
-
-
Adding SBT support to IntelliJ
-
open File > Settings > Plugin
-
-
click on Browser repositories
-
-
find SBT, right click and select Download and install
create folder ~/bin/ (if you dont have ~/bin in your PATH, then you can do this: echo "PATH=$PATH:~/bin" >> ~/.bashrc (refresh your currect session source ~/.bashrc)
Finally, now that you have all the tools you need, proceed to creating a new project
-
run the android plugin project setup tool: ~/bin/g8 jberkel/android-app, follow the onscreen questions (or press return for defaults) this will create the project folder and create the files needed to build an android project
My take on: How to give your new project a good starting point
So, you have a new shiny project on your lap, so now what? Today everyone is talking about “agile”, and how its better then the old “waterfall” methodology of doing things
-
Waterfall
The “Waterfall” approach is notorious in the software community, some highlights of why this is the case:
-
-
It assumes that the initial requirements will not change
-
There is only one version delivered to the stakeholders, at the end of the process
-
All the planning and “thinking” happen only in the beginning of the project
-
-
Agile
Everyone likes “Agile”, its the “cool” way to manage a project, it asks for less paperwork, less work upfront, built from the ground up to support changes, to name a few conceptions about it. While I call it “Agile” here, in reality it is a world of sub cultures, some of which are: “XP” (eXtreme Programming), “scrum”, “Kanban” to name a few. The difference between specific cultures is not always clear, we don’t necessarily have to stick to one, we can mix and match, like I am doing here, thus from this point onward I will just ignorantly call it “Agile” with out specifying the specific methodology. The main strengths going for the Agile methodology are
-
-
Short upfront work phase (requirements, design ..)
-
Iterations - project is broken down to deliverables, which is translated to real value delivered very early in the project lifetime
-
Changes are welcome and expected and are introduced into following iterations (bugs found in previous deliverables, design “inadequacies”)
-
-
Doing things in small pieces is great, just ask a WEB developer, how there’s nothing quite like writing a bit of code, refreshing the page and seeing the changes reflected, especially when working on a big complex project. This gives you a feeling of progress and achievement, even though the end goal is far far away..
-
Some Agile approaches talk beyond writing business code and going into proactive bug searching using TDD (Test Driven Development), which turns out to be a good fit to the overall process as it secures quality of delivered pieces of logic. BDD (Behavior Driven Development) is very similar to TDD, but instead of going after “test coverage” it takes on a higher level, business specification driven approach, where the business embodied user stories are part of the actual test suite.
-
My recipe for success
-
User Stories
-
Mock-ups (in the case of an app that has UI)
-
HLD (High Level -system component- Design)
-
Process Sequence diagram / Workflow diagram (possibly atop HLD)
-
-
Tooling
Tools that keep me productive:
-
-
Google Document (User Stories)
-
Google Drawing (mockups, diagrams, HLD)
-
Lucidchart (diagrams)
-
-
In addition, I use a light weight web based project management tool as a central place to keep reference to important docs and user stories, and see my progress (and others working on the project). A great tool for the job is Trello.com - which free and super simple!
The Kotlin project page has this to say about Kotlin: “is a statically typed programming language that compiles to JVM byte codes and JavaScript.” project page
-
Kotlin is a very interesting project, I wanted to check it out, so I decided to write this blog post exploring some design aspects behind Kotlin, and specifically how it compares to the leading JVM “hacker” language - Scala.
-
Who Is Behind Kotlin?
Even though there are several decent free IDE’s available today (Eclipse, NetBeans, Visual Studio Express..), it (could be) surprising that JetBrains is able to rack in between 199$ for a personal license and up to 699$ for a commercial license (for companies) including 1 year update subscription (I have indeed bought one personal license for my development needs). If you are developing on top of VisualStudio you are (most) probably familiar with the 250$+ plugin ReSharper.
-
I would argue that this makes them a good candidate, with insight into good and bad language features, but lets not get ahead of ourselves and set the expectations too high.
-
Why Another JVM Language?
JVM is a well known platform, its weaknesses and scalability strategies are available and documented well on the web, its being used as classic Java run-time VM and for some other popular languages like Scala, Groovy, Clojure and others. Many modules have been already written in Java, normally JVM run languages are compatible with the existing Java ecosystem.
-
Why Static Typing?
In the old (and still ongoing) battle of dynamic and static languages, the consensus is that dynamic languages are faster and easier way to get things done, on the static languages side they allow programmers harness the power of intelligent IDEs to scale up and maintain their code. Type-inferred languages try to find a middle ground approach so that you don’t need to write so much boilerplate code (cue JAVA) to support the static typing, while having the benefits of a full compile time type enforcement.
-
Finally, Kotlin Vs. Scala - Fight!
Scala today is the fastest growing JVM language and is probably competing for the same crowd as Kotlin, Kotlin put up a comparison page link so we will simply go over the omissions part and try to make sense of it.
-
OMITTED - Implicit conversions, parameters
What is it? Scala implicits is (IMHO) a single biggest weakness in the language, once they are used, along with the ability to boast symbols for syntax, it quickly stops being human readable and more like a DSL. Which requires many trips to the docs or even worse - the source code.
-
SBT (Scala Build Tool) is the Maven/Ant of the Scala world link, its very difficult to get things done with it without many trips to google/documentation
Lift Is a popular web framework, here’s a snippet I took from one of its documentation pages link (see the "#result *" #> x part)
-
1 2 3 4 5 6 7 8 9
defrender= inputParam match { caseFull(x) => println("Input is: "+x) "#result *" #> x case _ => println("No input present! Rendering input form HTML") PassThru }
-
Thoughts - Indeed for many people this is the part that makes Kotlin a better “scalable” language, by not allowing programmers and libraries hinder the code unreadable without an IDE
-
OMITTED - Overridable Type Members
What is it? Type members in Scala allow you to manage local generic types by assign a type to a class/abstract class/trait member
-
1 2 3
abstractclassHasTypedMember{ typetypeMe }
-
An extending class will provide a specific type for this type type member
-
1 2 3
classExtendingClassextendsHasTypedMember{ typetypeMe= Int }
-
Thoughts - Frankly, its not really clear why Kotlin decided to omit having typed members, I could be missing something.. Never the less Kotlin usually takes the minimalistic and simplistic route, my guess is that if this is indeed useful, the door is open to add it later..
-
OMITTED - Path-dependent types
What is it? Pointing to a class under another class in Java would mean a specific type (Parent.Child), well not so in Scala. Scala makes the nested class of two same class instances different, much like you would expect two members of two different instances of a single class. Confused? here is some code to explain this better
-
Java:
-
1 2 3 4 5 6 7 8 9 10 11 12
publicclassOuter{ publicclassInner{} publicvoiduseInner(Outer.Inner inner){...} } //.. somewhere in the code o1 = new Outer() o2 = new Outer() i1 = new o1.inner() i2 = new o2.inner() // this would work fine o1.useInner(i2) // works o2.useInner(i1) // works
-
Scala:
1 2 3 4 5 6 7 8 9 10 11 12
classOuter{ classInner{} defuseInner(inner : Inner){...} } //.. somewhere in the code.. val o1 = newOuter val o2 = newOuter val i1 = new o1.Inner val i2 = new o2.Inner o1.useInner(i1) // works fine o2.useInner(i2) // works fine o1.useInner(i2) // THROWS type mismatch exception!
-
Thoughts - It makes some sense that the path to a type that can only exist in an instance is enforced as such.. It would be interesting to know why Kotlin decided to implement this feature in a similar way to Java (conservative?)
-
OMITTED - Existential types
What is it? This is yet another trade-off for Scala to be able to inter-op with Java Let’s see an example:
-
1 2 3 4 5
// Create a case class with generic type caseclassZ[T]( blah:T) // create an instance and cast it to a type created with existential type annotation Seq( Z[Int]( 5 ), Z[String]("blah") ) : Seq[Z[X] forSome { typeX>: StringwithInt }] res1 = Seq[Z[_ >: StringwithInt]] = List(Z(5), Z(blah))
-
Thoughts - Scala supports type variance to describe generic type bounds, Scala designer (Odersky) chose to add existential types support due to three main reasons link: erasure, raw types (lack of generics) and Java’s “wildcard” types.
-
Kotlin took the more “purist” approach (also known as reified types) of dropping support for erasure, raw types, wildcard types and finally existential types.
-
OMITTED - Complicated logic for initialization of traits
What is it? Traits are similar to abstract classes, except that you may use several traits on the same class at the same time (like interfaces).
-
In Scala, traits are constructed before the extended classes
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14
traitAverager{ val first : Int val second : Int val avg : Int = first / second } // if we were to naively construct a class like so: classNaiveextendsAverager{ val first = 20 val second = 10 } // create instance > newNaive() java.lang.ArithmeticException: / by zero // what happened was that our val "avg" was computed at construction time before seeing the values we put in Naive
-
Alternatively the trait can be used with a pre-initialized anonymous class val res = new { val first = 20 ; val second = 10 } with Averager, also the val which depends on other vals (avg in our case) can be made “lazy” with the (obvious) lazy keyword lazy val avg : Int = first / second.
-
As a side note, using def for “avg” could work, but it would mean evaluating whatever is defined under “avg” every-time this method is called.
-
Thoughts - Traits in Scala can have state which could be made tricky. If you read this blog post link you will see Kotlin developers thoughts and a nice insight into what went into designing the inheritance in Kotlin. (don’t forget to read the comments!)
-
OMITTED - Custom symbolic operations
What is it? Scala does not have a built in handling of symbols (like =,+,/ etc), they are actually method calls, as you can have symbols used for method names.
-
Thoughts - This may sound useful at first, but can and (arguably) will create very confusing looking code, like we saw before in the Lift and SBT examples.
-
OMITTED - Built-in XML
What is it? Exactly as it sounds, Scala built in XML native support, so you can do stuff like this
-
1
val holdsXml = <outer><inner>some text</inner></outer>
-
Thoughts - Having a built in support for XML as a language feature is interesting, and sometimes useful. An interview link with Odersky reveals that he would consider removing the native XML support in Scala if had written Scala now.
-
Bottom Line
Kotlin is an idiomatic language that draws from the many good features in todays leading languages, specifically from Scala which is a leading language on the JVM platform today. We’ve looked into omissions on Kotlin’s part versus Scala
-
I’ve very much enjoyed writing this blog post and learning about the evolution that the design of Java/Scala/Kotlin languages, and the choices the respective authors made along the way.
-
To get a better feeling of Kotlin I might pick it up for a small project, at which point I will surely blog about it some more..
Contractor is a factory of contracts, that helps with documenting our APIs. The child_process facility built into node is using EventEmitter (v1), and does not support callbacks, so it is a great fit for Contractor! Since I needed it for a distributed worker framework I am working on, I added a helper facility called “Lawyer”, it basically reads the contracts and routes them to the provided object.
-
Let us take the parent and child example code from Node’s official documentation (http://nodejs.org/api/child_process.html) and see how simple it would be to add some documentation.
Node’s child_process facility provied us with a very basic means of communication, we simply call n.send({any: "thing"}) and it automagically recieved by the forked child. Let’s spice it up with a more RPC message passing style, with contracts and lawyers
var contracts = require('./contracts'); var Contractor = require('contractor').Contractor; var Lawyer = require('contractor').Lawyer; var cp = require('child_process');
“Strong typing is for people with weak memories” - not sure who originally said that, but you have got to love the enthusiasm people have for dynamic languges. What is there not to love? powerful basic language constructs such as closure, prototypal inheritance, objects as first class citizens and anonymous functions, mainly, with which you can simulate almost anything static languages have to offer!
-
Proof you say? no problem! here are few implementations that are worthy of noting:
(if you have other interesting examples let me know!)
-
Convention over configuration
The combination of CoffeeScript’s “classical” Class like system and Backbone.JS’s MVC style structure, give (IMHO) the best value in terms of complexity and pure fun writing apps. The challenges I come across while building large complex applications is the inherited nature of a true dynamic language “typelessness“.
-
The ugly monster, that is -a large application written in JavaScript, pushed me into struggling with these two main challenges:
-
-
Many-to-many PubSub communication
-
3rd party API’s
-
-
Many-to-many PubSub communication
While the PubSub (also known as observer pattern) in JavaScript is a powerful tool to decouple your code, having multiple callers and listeners on the same event makes it really difficult to keep track of the actual common API for these events. Lats try to reflect on this while looking at an actual implementation.
-
(please excuse my CoffeeScript)
-
Pub/Sub implementation in Backbone/Underscore.JS
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14
pubsub = _.extend {}, Backbone.Events pubsub.on 'myEvent', (param1, param2)-> x1 = param1 + param2 # do something with x1
# in some other obscure location in the code pubsub.on 'myEvent', (param1)-> x2 = param1 # do something with x2
The “pubsub.on(event name, callback)” API is very common amongst different JS pub/sub frameworks. here’s an example of how a generic implementation might look like:
If your using the very popular Socket.IO library, for realtime client/server communication, you are using the exact same API, in which case it makes it even more complicated to keep track of, since the ambiguity is manifested on both the client and the server.
-
3rd party API’s
This is how a type related bug might look like in a dynamic language
-
1 2 3
a = "42" # returned from some API a == 42 => false
I’m introducing a convention here, that helps keep the API consistent across the application, the approach here is to create wrapper functions, that are used as both the API documentation and run-time validation facility for required and optional arguments.
-
Example:
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
# declare the events and their APIs Messages = { LoginEvent : Contractor.Create( "LoginEvent", Contractor.Required("user ID"), Contractor.Optional("additional info") ) AppErrorEvent : Contractor.Create( "AppErrorEvent", Contractor.Required("user ID"), Contractor.Required("error description"), Contractor.Optional("exception") ) }
# register on event API pubsub.on Messages.LoginEvent, (userID, info)-> #... do stuff here pubsub.on Messages.AppErrorEvent, (userID, description, exception)-> #... do stuff here
# trigger the event pubsub.apply(pubsub, Messages.LoginEvent(112233, "mobile")) pubsub.apply(pubsub, Messages.LoginEvent(112233))
pubsub.apply(pubsub, Messages.AppErrorEvent(112233, "user did not have access to resource", e)) pubsub.apply(pubsub, Messages.AppErrorEvent(112233)) # since we did not provide the second required argument, this will log an error and return null
-
With NodeJS, I am using the same code on the front and back end, which makes it really easy to manage my Socket.IO communication.
-
If you have been paying attention, there’s still the type issue. Since I work allot with Backbone, it made a lot of sense to write “backbone-typed” which adds optional typing to Backbone-Models. Example:
from the list select the Android version your afer, right now its Android 4.0.3 (API 15) (I also installed Android 2.3.3), click on Install packages to get the installation started
-
-
setup your env echo "export ANDROID_HOME=~/Applications/android-sdk-linux/" >> ~/.bashrc
-
-
refresh your currect session source ~/.bashrc
-
-
-
Adding SBT support to IntelliJ
-
open File > Settings > Plugin
-
-
click on Browser repositories
-
-
find SBT, right click and select Download and install
create folder ~/bin/ (if you dont have ~/bin in your PATH, then you can do this: echo "PATH=$PATH:~/bin" >> ~/.bashrc (refresh your currect session source ~/.bashrc)
Finally, now that you have all the tools you need, proceed to creating a new project
-
run the android plugin project setup tool: ~/bin/g8 jberkel/android-app, follow the onscreen questions (or press return for defaults) this will create the project folder and create the files needed to build an android project
System Architect - mostly interested in the software stack, deployment and the database design
-
Team Members - mostly interested in the more lower level stuff like modules and API’s
-
Other Teams - mostly interested in API’s
-
-
Feedback on design is great for everyone involved, setup meetings with relevant parties to do DR (design review)
-
UML and Other Vegetables
UML is complex, you don’t need to know it perfectly, start with what makes sense, and slowly pick up tutorials. Remember, its a visual tool that helps you communicate, not everyone who will look at your designs has the time (or cares for) to read throughly a book about UML and its specifics. In Einstein’s words “Make everything as simple as possible, but not simpler.”
-
Disclosure
Design does not last. Nobody is going to update the design as the system grows and evolves. Having said that, can still be used a good reference.
“User Story” is an important communication tool, it is used between you (the developer) and who ever is making the requirements (the stakeholder). A “contract” which (unlike one written by a lawyer) is clear, understood and agreed upon. On our side (development) user stories embody a work unit that can be further broken down and estimated.
-
Key value points:
-
-
Language - its key that user stories are written in a language everyone can understand (stakeholders, developers etc..)
-
Perspective - user stories need to be written from the perspective of the “feature user”, so they represent real world value
-
Estimation - to help you get better and more precise estimation for the project overall
-
Prioritization - what needs to be delivered soon/later
-
-
Lets imagine we just got a call from the product manager, where he is telling us about this new survey tool:
-
It needs to collect answers from users via a link sent to our users by e-mail
-
It needs to be summarized and sent to … once a month on the 1st
-
Managers need to be able to see past surveys
-
-
Hmm.. OK, thats pretty clear, but suppose this can be written clearer, lats try using the following template: (not my invention;)) As a [role], I want [Requirement / Feature], So that [Justigication/ Reason]
-
The requirements now look like this:
-
-
As an end user, I will be able to follow a link I got via e-mail, so that I can fill a survey and express my opinion
-
As a manager, I will receive an e-mail with a digest of surveys, so that I can evaluate performance
-
As a manager, I will be able to access old surveys, so that I can compare and evaluate performance over time
-
-
By reading the above I can clearly understand the required functionality and the motivation behind writing it, this gives me a good starting point. Since I am the one to implement this, I will need to provide estimates, so that product can plan ahead and communicate the product road map onward. Unfortunately, “The Devil is In The Details” as they say, so giving estimates based on points 1,2,3 is still difficult. Points 2 & 3 are “huge” stories, also known as “epics”, we need to further break down these stories, while still using the same language and practices:
-
-
As an end user, I will receive an email with a link as a result of some communication with a company, so that I can follow it and fill a survey
-
As an end user, I get a set of questions to answer, so that my opinion is collected
-
As a manager, I will receive an e-mail with a digest of surveys, so that I can evaluate companies performance
-
As a manager, I will have a survey management portal, so that I can overview previous results and compare to
-
As a manager, I can select a specific survey, so that I can extract meaningful actionable information
-
-
The functionality expected of deliverables from the points above is clear, and now I feel more comfortable estimating the time it will take me to complete each. Along with the estimation, these can now be prioritized against each other and against other ongoing projects..
-
Summery
The beginning of a new project is the perfect time to introduce user stories, it will help with getting the conversation going, further clarifying the scope and requirements. While the project scope might change, user stories are (optimally) modular, decoupled and encapsulated, this should help with managing changes and their effect on the overall project.
My take on: How to give your new project a good starting point
So, you have a new shiny project on your lap, so now what? Today everyone is talking about “agile”, and how its better then the old “waterfall” methodology of doing things
-
Waterfall
The “Waterfall” approach is notorious in the software community, some highlights of why this is the case:
-
-
It assumes that the initial requirements will not change
-
There is only one version delivered to the stakeholders, at the end of the process
-
All the planning and “thinking” happen only in the beginning of the project
-
-
Agile
Everyone likes “Agile”, its the “cool” way to manage a project, it asks for less paperwork, less work upfront, built from the ground up to support changes, to name a few conceptions about it. While I call it “Agile” here, in reality it is a world of sub cultures, some of which are: “XP” (eXtreme Programming), “scrum”, “Kanban” to name a few. The difference between specific cultures is not always clear, we don’t necessarily have to stick to one, we can mix and match, like I am doing here, thus from this point onward I will just ignorantly call it “Agile” with out specifying the specific methodology. The main strengths going for the Agile methodology are
-
-
Short upfront work phase (requirements, design ..)
-
Iterations - project is broken down to deliverables, which is translated to real value delivered very early in the project lifetime
-
Changes are welcome and expected and are introduced into following iterations (bugs found in previous deliverables, design “inadequacies”)
-
-
Doing things in small pieces is great, just ask a WEB developer, how there’s nothing quite like writing a bit of code, refreshing the page and seeing the changes reflected, especially when working on a big complex project. This gives you a feeling of progress and achievement, even though the end goal is far far away..
-
Some Agile approaches talk beyond writing business code and going into proactive bug searching using TDD (Test Driven Development), which turns out to be a good fit to the overall process as it secures quality of delivered pieces of logic. BDD (Behavior Driven Development) is very similar to TDD, but instead of going after “test coverage” it takes on a higher level, business specification driven approach, where the business embodied user stories are part of the actual test suite.
-
My recipe for success
-
User Stories
-
Mock-ups (in the case of an app that has UI)
-
HLD (High Level -system component- Design)
-
Process Sequence diagram / Workflow diagram (possibly atop HLD)
-
-
Tooling
Tools that keep me productive:
-
-
Google Document (User Stories)
-
Google Drawing (mockups, diagrams, HLD)
-
Lucidchart (diagrams)
-
-
In addition, I use a light weight web based project management tool as a central place to keep reference to important docs and user stories, and see my progress (and others working on the project). A great tool for the job is Trello.com - which free and super simple!
If you need to test/mock 3rd party API’s / a crawler / external web service, and you dont want to use some fancy DI framework (I hate the magic in DI frameworks, I use constructor DI instead..), anyway, here’s a helper class I put togather to that end
-
If you haven’t got a chance to try out Spray, you totally should, its a high performance web server built on top of Netty and Akka
-
Spray has a very nice HTTP API. Few things I liked:
-
-
HTTP method mapping spray.http.HttpMethods - GET, POST etc..
-
HttpRequest and HttpResopnse are basically wrappers around data you would expect to have in a request and response data
-
spray.http.Uri wrapper class around what you would expect to find in the URI
-
it really is just an Akka Actor, with just the right amount of control exposed and the rest well hidden behind its nice APIs
import akka.actor._ import akka.io.IO import java.util.concurrent.atomic.AtomicInteger import spray.can.Http import spray.http.{Uri, HttpMethod, HttpResponse, HttpRequest} objectHelpers{ lazyimplicitval testSystem = ActorSystem("helpers-test-system") // used to track the last assigned port val lastPort = newAtomicInteger(20000) // a single expected triplet of method+uri+body typeReplay= (HttpMethod,Uri.Path,String) // factory method to get a replater to play around with defwebReplayer(replay : List[Replay]) : (ActorRef,Int) = { // everytime the factory is used a new port is assigned, // so many instances can co-exist and tests can run in parallel without getting // on eachother's toes val newPort = lastPort.addAndGet(1) // instantiate a new Spray server. Yes, its that simple! val newServer = testSystem.actorOf(Props(newHelpers.ReplayerActor(replay))) IO(Http) ! Http.Bind(newServer, interface = "127.0.0.1", port = newPort) // return a pair of the server instance and the assigned port // the newPort should be used by client code to construct the remote URI with (newServer,newPort) } // object to use to tell the server to shutdown.. objectShutTestServer // a concrete implementation for a Spray Actor to listen to incoming HttpRequest(..) messages // provided a list of messages to be matched against arrived and content to be sent back classReplayerActor(var replay :List[Replay]) extendsActor{ var ioServer : ActorRef = null overridedefreceive: Receive = { case _ : Http.Connected => sender ! Http.Register(self) ioServer = sender() // this is basically a "catch all" HttpRequest pattern caseHttpRequest(method, uri,_,_,_) => // a test to see that this is indeed expected to happen at this point if (!replay.isEmpty && replay.head._1 == method && uri.path == replay.head._2){ sender ! HttpResponse(entity = replay.head._3) replay = replay.tail } else { sender ! HttpResponse(status = 400, entity = "was expecting a different request at this point") } caseShutTestServer => ioServer ! PoisonPill case _ => //ignore } } }
-
And now a cocrete client code example to use this tool
// some random content I would like to receive in this scenario (or test..) val content = s""" |<html> |<head> |<title>sometitle</title> |</head> |<body>some body</body> |</html> """.stripMargin // we use the factory method to get an instance of "replayer" to play around with val (_testWebServer,port) = Helpers.webReplayer( List( (GET,Uri.Path("/somesite"),content) ) ) // and where ever in your actual code, use the port we got back above and have it "hit" the server val whatEver = YourHTTPGetter.fetch(s"http://127.0.0.1:$port/somesite") assert(whatEver == content)
The Kotlin project page has this to say about Kotlin: “is a statically typed programming language that compiles to JVM byte codes and JavaScript.” project page
-
Kotlin is a very interesting project, I wanted to check it out, so I decided to write this blog post exploring some design aspects behind Kotlin, and specifically how it compares to the leading JVM “hacker” language - Scala.
-
Who Is Behind Kotlin?
Even though there are several decent free IDE’s available today (Eclipse, NetBeans, Visual Studio Express..), it (could be) surprising that JetBrains is able to rack in between 199$ for a personal license and up to 699$ for a commercial license (for companies) including 1 year update subscription (I have indeed bought one personal license for my development needs). If you are developing on top of VisualStudio you are (most) probably familiar with the 250$+ plugin ReSharper.
-
I would argue that this makes them a good candidate, with insight into good and bad language features, but lets not get ahead of ourselves and set the expectations too high.
-
Why Another JVM Language?
JVM is a well known platform, its weaknesses and scalability strategies are available and documented well on the web, its being used as classic Java run-time VM and for some other popular languages like Scala, Groovy, Clojure and others. Many modules have been already written in Java, normally JVM run languages are compatible with the existing Java ecosystem.
-
Why Static Typing?
In the old (and still ongoing) battle of dynamic and static languages, the consensus is that dynamic languages are faster and easier way to get things done, on the static languages side they allow programmers harness the power of intelligent IDEs to scale up and maintain their code. Type-inferred languages try to find a middle ground approach so that you don’t need to write so much boilerplate code (cue JAVA) to support the static typing, while having the benefits of a full compile time type enforcement.
-
Finally, Kotlin Vs. Scala - Fight!
Scala today is the fastest growing JVM language and is probably competing for the same crowd as Kotlin, Kotlin put up a comparison page link so we will simply go over the omissions part and try to make sense of it.
-
OMITTED - Implicit conversions, parameters
What is it? Scala implicits is (IMHO) a single biggest weakness in the language, once they are used, along with the ability to boast symbols for syntax, it quickly stops being human readable and more like a DSL. Which requires many trips to the docs or even worse - the source code.
-
SBT (Scala Build Tool) is the Maven/Ant of the Scala world link, its very difficult to get things done with it without many trips to google/documentation
Lift Is a popular web framework, here’s a snippet I took from one of its documentation pages link (see the "#result *" #> x part)
-
1 2 3 4 5 6 7 8 9
defrender= inputParam match { caseFull(x) => println("Input is: "+x) "#result *" #> x case _ => println("No input present! Rendering input form HTML") PassThru }
-
Thoughts - Indeed for many people this is the part that makes Kotlin a better “scalable” language, by not allowing programmers and libraries hinder the code unreadable without an IDE
-
OMITTED - Overridable Type Members
What is it? Type members in Scala allow you to manage local generic types by assign a type to a class/abstract class/trait member
-
1 2 3
abstractclassHasTypedMember{ typetypeMe }
-
An extending class will provide a specific type for this type type member
-
1 2 3
classExtendingClassextendsHasTypedMember{ typetypeMe= Int }
-
Thoughts - Frankly, its not really clear why Kotlin decided to omit having typed members, I could be missing something.. Never the less Kotlin usually takes the minimalistic and simplistic route, my guess is that if this is indeed useful, the door is open to add it later..
-
OMITTED - Path-dependent types
What is it? Pointing to a class under another class in Java would mean a specific type (Parent.Child), well not so in Scala. Scala makes the nested class of two same class instances different, much like you would expect two members of two different instances of a single class. Confused? here is some code to explain this better
-
Java:
-
1 2 3 4 5 6 7 8 9 10 11 12
publicclassOuter{ publicclassInner{} publicvoiduseInner(Outer.Inner inner){...} } //.. somewhere in the code o1 = new Outer() o2 = new Outer() i1 = new o1.inner() i2 = new o2.inner() // this would work fine o1.useInner(i2) // works o2.useInner(i1) // works
-
Scala:
1 2 3 4 5 6 7 8 9 10 11 12
classOuter{ classInner{} defuseInner(inner : Inner){...} } //.. somewhere in the code.. val o1 = newOuter val o2 = newOuter val i1 = new o1.Inner val i2 = new o2.Inner o1.useInner(i1) // works fine o2.useInner(i2) // works fine o1.useInner(i2) // THROWS type mismatch exception!
-
Thoughts - It makes some sense that the path to a type that can only exist in an instance is enforced as such.. It would be interesting to know why Kotlin decided to implement this feature in a similar way to Java (conservative?)
-
OMITTED - Existential types
What is it? This is yet another trade-off for Scala to be able to inter-op with Java Let’s see an example:
-
1 2 3 4 5
// Create a case class with generic type caseclassZ[T]( blah:T) // create an instance and cast it to a type created with existential type annotation Seq( Z[Int]( 5 ), Z[String]("blah") ) : Seq[Z[X] forSome { typeX>: StringwithInt }] res1 = Seq[Z[_ >: StringwithInt]] = List(Z(5), Z(blah))
-
Thoughts - Scala supports type variance to describe generic type bounds, Scala designer (Odersky) chose to add existential types support due to three main reasons link: erasure, raw types (lack of generics) and Java’s “wildcard” types.
-
Kotlin took the more “purist” approach (also known as reified types) of dropping support for erasure, raw types, wildcard types and finally existential types.
-
OMITTED - Complicated logic for initialization of traits
What is it? Traits are similar to abstract classes, except that you may use several traits on the same class at the same time (like interfaces).
-
In Scala, traits are constructed before the extended classes
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14
traitAverager{ val first : Int val second : Int val avg : Int = first / second } // if we were to naively construct a class like so: classNaiveextendsAverager{ val first = 20 val second = 10 } // create instance > newNaive() java.lang.ArithmeticException: / by zero // what happened was that our val "avg" was computed at construction time before seeing the values we put in Naive
-
Alternatively the trait can be used with a pre-initialized anonymous class val res = new { val first = 20 ; val second = 10 } with Averager, also the val which depends on other vals (avg in our case) can be made “lazy” with the (obvious) lazy keyword lazy val avg : Int = first / second.
-
As a side note, using def for “avg” could work, but it would mean evaluating whatever is defined under “avg” every-time this method is called.
-
Thoughts - Traits in Scala can have state which could be made tricky. If you read this blog post link you will see Kotlin developers thoughts and a nice insight into what went into designing the inheritance in Kotlin. (don’t forget to read the comments!)
-
OMITTED - Custom symbolic operations
What is it? Scala does not have a built in handling of symbols (like =,+,/ etc), they are actually method calls, as you can have symbols used for method names.
-
Thoughts - This may sound useful at first, but can and (arguably) will create very confusing looking code, like we saw before in the Lift and SBT examples.
-
OMITTED - Built-in XML
What is it? Exactly as it sounds, Scala built in XML native support, so you can do stuff like this
-
1
val holdsXml = <outer><inner>some text</inner></outer>
-
Thoughts - Having a built in support for XML as a language feature is interesting, and sometimes useful. An interview link with Odersky reveals that he would consider removing the native XML support in Scala if had written Scala now.
-
Bottom Line
Kotlin is an idiomatic language that draws from the many good features in todays leading languages, specifically from Scala which is a leading language on the JVM platform today. We’ve looked into omissions on Kotlin’s part versus Scala
-
I’ve very much enjoyed writing this blog post and learning about the evolution that the design of Java/Scala/Kotlin languages, and the choices the respective authors made along the way.
-
To get a better feeling of Kotlin I might pick it up for a small project, at which point I will surely blog about it some more..
Unlike the classic FIFO queue, jobs are scheduled for execution in the future, possibly on a specific day, week, month year.. similar to “cron” in a Unix OS. Unlike the normal Unix “cron” implementation, we want it to be distributed, so that several machines(workers) can query and then execute these jobs.
-
Implementing
-
Planning - The standart Unix cron format is very flexible and has been time tested for describing schedules
-
Managing/Coordinating - The queues are stored in Redis Sorted Sets, Lua will be used for atomic execution
-
Execution - Getting the pieces together
-
-
Planning
“Unix Time” format, is the number of milliseconds since (UTC) 1 January 1970 Example (* the actual unix time number is in UTC, while the formatted date is localized)
-
1350745504313 == Sat Oct 20 2012 17:05:04 GMT+0200 (IST)
-
Using this format will make it possible to slice the queue with specific point in time in mind.
-
Managing/Coordinating
Redis’st will be holding the job IDs as members scored by their corresponding execution times(in unix time format).
Finding a job to run: ZRANGEBYSCORE cron:queue 1350745504314 LIMIT 0 1
-
-
Execution
Without going into any platform specific implementation details, this is what the process looks like
-
-
When a new schedule is added, jobs are created for upcoming execution times, which are calculated for the next two hours and added to our cron queue
-
A special job is added to be run every hour, it will run over all the schedules, calculate and add their execution time to the scheduled queue, two hours into the future
-
Workers will be polling the database every second to find jobs
-
Once an entry comes up, it is removed from the queue and added to another Sorted Set queue, so we dont “forget” any jobs..
-
-
As for items 3 and 4 above, it is not possible to implement this logic in the client, because for example when we find a job that needs to be executed, another worker might grab it, or worse they both “grabbed” it and are now working on the same job. To work around this, we need to write a little Lua script, since Lua scripts are executed serially, just like any other Redis command
local currentTimeMS = KEYS[1] local workingQueue = "scheduler:working" local scheduleQueue = "schedule" -- for easier reading, epoch time is in milliseconds local second = 1000
localfunctionfoundJob(jobID) -- implement removing the job from all other queues -- implement updating job status where the actual job object is stored end
localfunctionfindNextJob() local result -- check if there are any "stale" jobs result = redis.call( 'ZRANGEBYSCORE', workingQueue, 0, currentTimeMS-60*second, 'LIMIT',0,1 ) if result[1] ~= nilthenreturn foundJob(result[1]) end -- check the normal queue result = redis.call( 'ZRANGEBYSCORE', scheduleQueue, 0, currentTimeMS, 'LIMIT',0,1 ) if result[1] ~= nilthenreturn foundJob(result[1]) end -- if we didnt find anything return nil returnnil end
return findNextJob()
-
And there you have it! a “Distributed scheduled queue” implemented on top of Redis.
-
If you are going to implement this with node, I created a little cron format parser you can find here: JSCron
Contractor is a factory of contracts, that helps with documenting our APIs. The child_process facility built into node is using EventEmitter (v1), and does not support callbacks, so it is a great fit for Contractor! Since I needed it for a distributed worker framework I am working on, I added a helper facility called “Lawyer”, it basically reads the contracts and routes them to the provided object.
-
Let us take the parent and child example code from Node’s official documentation (http://nodejs.org/api/child_process.html) and see how simple it would be to add some documentation.
Node’s child_process facility provied us with a very basic means of communication, we simply call n.send({any: "thing"}) and it automagically recieved by the forked child. Let’s spice it up with a more RPC message passing style, with contracts and lawyers
var contracts = require('./contracts'); var Contractor = require('contractor').Contractor; var Lawyer = require('contractor').Lawyer; var cp = require('child_process');
“Strong typing is for people with weak memories” - not sure who originally said that, but you have got to love the enthusiasm people have for dynamic languges. What is there not to love? powerful basic language constructs such as closure, prototypal inheritance, objects as first class citizens and anonymous functions, mainly, with which you can simulate almost anything static languages have to offer!
-
Proof you say? no problem! here are few implementations that are worthy of noting:
(if you have other interesting examples let me know!)
-
Convention over configuration
The combination of CoffeeScript’s “classical” Class like system and Backbone.JS’s MVC style structure, give (IMHO) the best value in terms of complexity and pure fun writing apps. The challenges I come across while building large complex applications is the inherited nature of a true dynamic language “typelessness“.
-
The ugly monster, that is -a large application written in JavaScript, pushed me into struggling with these two main challenges:
-
-
Many-to-many PubSub communication
-
3rd party API’s
-
-
Many-to-many PubSub communication
While the PubSub (also known as observer pattern) in JavaScript is a powerful tool to decouple your code, having multiple callers and listeners on the same event makes it really difficult to keep track of the actual common API for these events. Lats try to reflect on this while looking at an actual implementation.
-
(please excuse my CoffeeScript)
-
Pub/Sub implementation in Backbone/Underscore.JS
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14
pubsub = _.extend {}, Backbone.Events pubsub.on 'myEvent', (param1, param2)-> x1 = param1 + param2 # do something with x1
# in some other obscure location in the code pubsub.on 'myEvent', (param1)-> x2 = param1 # do something with x2
The “pubsub.on(event name, callback)” API is very common amongst different JS pub/sub frameworks. here’s an example of how a generic implementation might look like:
If your using the very popular Socket.IO library, for realtime client/server communication, you are using the exact same API, in which case it makes it even more complicated to keep track of, since the ambiguity is manifested on both the client and the server.
-
3rd party API’s
This is how a type related bug might look like in a dynamic language
-
1 2 3
a = "42" # returned from some API a == 42 => false
I’m introducing a convention here, that helps keep the API consistent across the application, the approach here is to create wrapper functions, that are used as both the API documentation and run-time validation facility for required and optional arguments.
-
Example:
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
# declare the events and their APIs Messages = { LoginEvent : Contractor.Create( "LoginEvent", Contractor.Required("user ID"), Contractor.Optional("additional info") ) AppErrorEvent : Contractor.Create( "AppErrorEvent", Contractor.Required("user ID"), Contractor.Required("error description"), Contractor.Optional("exception") ) }
# register on event API pubsub.on Messages.LoginEvent, (userID, info)-> #... do stuff here pubsub.on Messages.AppErrorEvent, (userID, description, exception)-> #... do stuff here
# trigger the event pubsub.apply(pubsub, Messages.LoginEvent(112233, "mobile")) pubsub.apply(pubsub, Messages.LoginEvent(112233))
pubsub.apply(pubsub, Messages.AppErrorEvent(112233, "user did not have access to resource", e)) pubsub.apply(pubsub, Messages.AppErrorEvent(112233)) # since we did not provide the second required argument, this will log an error and return null
-
With NodeJS, I am using the same code on the front and back end, which makes it really easy to manage my Socket.IO communication.
-
If you have been paying attention, there’s still the type issue. Since I work allot with Backbone, it made a lot of sense to write “backbone-typed” which adds optional typing to Backbone-Models. Example:
System Architect - mostly interested in the software stack, deployment and the database design
-
Team Members - mostly interested in the more lower level stuff like modules and API’s
-
Other Teams - mostly interested in API’s
-
-
Feedback on design is great for everyone involved, setup meetings with relevant parties to do DR (design review)
-
UML and Other Vegetables
UML is complex, you don’t need to know it perfectly, start with what makes sense, and slowly pick up tutorials. Remember, its a visual tool that helps you communicate, not everyone who will look at your designs has the time (or cares for) to read throughly a book about UML and its specifics. In Einstein’s words “Make everything as simple as possible, but not simpler.”
-
Disclosure
Design does not last. Nobody is going to update the design as the system grows and evolves. Having said that, can still be used a good reference.
“User Story” is an important communication tool, it is used between you (the developer) and who ever is making the requirements (the stakeholder). A “contract” which (unlike one written by a lawyer) is clear, understood and agreed upon. On our side (development) user stories embody a work unit that can be further broken down and estimated.
-
Key value points:
-
-
Language - its key that user stories are written in a language everyone can understand (stakeholders, developers etc..)
-
Perspective - user stories need to be written from the perspective of the “feature user”, so they represent real world value
-
Estimation - to help you get better and more precise estimation for the project overall
-
Prioritization - what needs to be delivered soon/later
-
-
Lets imagine we just got a call from the product manager, where he is telling us about this new survey tool:
-
It needs to collect answers from users via a link sent to our users by e-mail
-
It needs to be summarized and sent to … once a month on the 1st
-
Managers need to be able to see past surveys
-
-
Hmm.. OK, thats pretty clear, but suppose this can be written clearer, lats try using the following template: (not my invention;)) As a [role], I want [Requirement / Feature], So that [Justigication/ Reason]
-
The requirements now look like this:
-
-
As an end user, I will be able to follow a link I got via e-mail, so that I can fill a survey and express my opinion
-
As a manager, I will receive an e-mail with a digest of surveys, so that I can evaluate performance
-
As a manager, I will be able to access old surveys, so that I can compare and evaluate performance over time
-
-
By reading the above I can clearly understand the required functionality and the motivation behind writing it, this gives me a good starting point. Since I am the one to implement this, I will need to provide estimates, so that product can plan ahead and communicate the product road map onward. Unfortunately, “The Devil is In The Details” as they say, so giving estimates based on points 1,2,3 is still difficult. Points 2 & 3 are “huge” stories, also known as “epics”, we need to further break down these stories, while still using the same language and practices:
-
-
As an end user, I will receive an email with a link as a result of some communication with a company, so that I can follow it and fill a survey
-
As an end user, I get a set of questions to answer, so that my opinion is collected
-
As a manager, I will receive an e-mail with a digest of surveys, so that I can evaluate companies performance
-
As a manager, I will have a survey management portal, so that I can overview previous results and compare to
-
As a manager, I can select a specific survey, so that I can extract meaningful actionable information
-
-
The functionality expected of deliverables from the points above is clear, and now I feel more comfortable estimating the time it will take me to complete each. Along with the estimation, these can now be prioritized against each other and against other ongoing projects..
-
Summery
The beginning of a new project is the perfect time to introduce user stories, it will help with getting the conversation going, further clarifying the scope and requirements. While the project scope might change, user stories are (optimally) modular, decoupled and encapsulated, this should help with managing changes and their effect on the overall project.
from the list select the Android version your afer, right now its Android 4.0.3 (API 15) (I also installed Android 2.3.3), click on Install packages to get the installation started
-
-
setup your env echo "export ANDROID_HOME=~/Applications/android-sdk-linux/" >> ~/.bashrc
-
-
refresh your currect session source ~/.bashrc
-
-
-
Adding SBT support to IntelliJ
-
open File > Settings > Plugin
-
-
click on Browser repositories
-
-
find SBT, right click and select Download and install
create folder ~/bin/ (if you dont have ~/bin in your PATH, then you can do this: echo "PATH=$PATH:~/bin" >> ~/.bashrc (refresh your currect session source ~/.bashrc)
Finally, now that you have all the tools you need, proceed to creating a new project
-
run the android plugin project setup tool: ~/bin/g8 jberkel/android-app, follow the onscreen questions (or press return for defaults) this will create the project folder and create the files needed to build an android project
My take on: How to give your new project a good starting point
So, you have a new shiny project on your lap, so now what? Today everyone is talking about “agile”, and how its better then the old “waterfall” methodology of doing things
-
Waterfall
The “Waterfall” approach is notorious in the software community, some highlights of why this is the case:
-
-
It assumes that the initial requirements will not change
-
There is only one version delivered to the stakeholders, at the end of the process
-
All the planning and “thinking” happen only in the beginning of the project
-
-
Agile
Everyone likes “Agile”, its the “cool” way to manage a project, it asks for less paperwork, less work upfront, built from the ground up to support changes, to name a few conceptions about it. While I call it “Agile” here, in reality it is a world of sub cultures, some of which are: “XP” (eXtreme Programming), “scrum”, “Kanban” to name a few. The difference between specific cultures is not always clear, we don’t necessarily have to stick to one, we can mix and match, like I am doing here, thus from this point onward I will just ignorantly call it “Agile” with out specifying the specific methodology. The main strengths going for the Agile methodology are
-
-
Short upfront work phase (requirements, design ..)
-
Iterations - project is broken down to deliverables, which is translated to real value delivered very early in the project lifetime
-
Changes are welcome and expected and are introduced into following iterations (bugs found in previous deliverables, design “inadequacies”)
-
-
Doing things in small pieces is great, just ask a WEB developer, how there’s nothing quite like writing a bit of code, refreshing the page and seeing the changes reflected, especially when working on a big complex project. This gives you a feeling of progress and achievement, even though the end goal is far far away..
-
Some Agile approaches talk beyond writing business code and going into proactive bug searching using TDD (Test Driven Development), which turns out to be a good fit to the overall process as it secures quality of delivered pieces of logic. BDD (Behavior Driven Development) is very similar to TDD, but instead of going after “test coverage” it takes on a higher level, business specification driven approach, where the business embodied user stories are part of the actual test suite.
-
My recipe for success
-
User Stories
-
Mock-ups (in the case of an app that has UI)
-
HLD (High Level -system component- Design)
-
Process Sequence diagram / Workflow diagram (possibly atop HLD)
-
-
Tooling
Tools that keep me productive:
-
-
Google Document (User Stories)
-
Google Drawing (mockups, diagrams, HLD)
-
Lucidchart (diagrams)
-
-
In addition, I use a light weight web based project management tool as a central place to keep reference to important docs and user stories, and see my progress (and others working on the project). A great tool for the job is Trello.com - which free and super simple!
diff --git a/package.json b/package.json
new file mode 100644
index 0000000..809afdd
--- /dev/null
+++ b/package.json
@@ -0,0 +1,20 @@
+{
+ "name": "hexo-site",
+ "version": "0.0.0",
+ "private": true,
+ "hexo": {
+ "version": "3.2.0"
+ },
+ "dependencies": {
+ "hexo": "^3.1.1",
+ "hexo-deployer-git": "^0.1.0",
+ "hexo-generator-archive": "^0.1.2",
+ "hexo-generator-category": "^0.1.2",
+ "hexo-generator-index": "^0.1.2",
+ "hexo-generator-tag": "^0.1.1",
+ "hexo-renderer-ejs": "^0.1.0",
+ "hexo-renderer-marked": "^0.2.4",
+ "hexo-renderer-stylus": "^0.3.0",
+ "hexo-server": "^0.1.2"
+ }
+}
diff --git a/scaffolds/draft.md b/scaffolds/draft.md
new file mode 100644
index 0000000..45b1bb7
--- /dev/null
+++ b/scaffolds/draft.md
@@ -0,0 +1,3 @@
+title: {{ title }}
+tags:
+---
diff --git a/scaffolds/page.md b/scaffolds/page.md
new file mode 100644
index 0000000..f484b76
--- /dev/null
+++ b/scaffolds/page.md
@@ -0,0 +1,3 @@
+title: {{ title }}
+date: {{ date }}
+---
diff --git a/scaffolds/photo.md b/scaffolds/photo.md
new file mode 100644
index 0000000..5aba0db
--- /dev/null
+++ b/scaffolds/photo.md
@@ -0,0 +1,5 @@
+layout: {{ layout }}
+title: {{ title }}
+date: {{ date }}
+tags:
+---
diff --git a/scaffolds/post.md b/scaffolds/post.md
new file mode 100644
index 0000000..c590d7a
--- /dev/null
+++ b/scaffolds/post.md
@@ -0,0 +1,4 @@
+title: {{ title }}
+date: {{ date }}
+tags:
+---
diff --git a/source/_posts/Using-Spray-to-mock-3rd-party-APIs-in-your-tests.md b/source/_posts/Using-Spray-to-mock-3rd-party-APIs-in-your-tests.md
new file mode 100644
index 0000000..2c3003a
--- /dev/null
+++ b/source/_posts/Using-Spray-to-mock-3rd-party-APIs-in-your-tests.md
@@ -0,0 +1,105 @@
+title: 'Using Spray to mock 3rd party APIs in your tests'
+date: 2014-06-14 20:26:44
+tags: ["scala","spray","testing"]
+---
+
+If you need to test/mock 3rd party API's / a crawler / external web service, and you dont want to use some fancy DI framework (I hate the magic in DI frameworks, I use constructor DI instead..), anyway, here's a helper class I put togather to that end
+
+If you haven't got a chance to try out Spray, you totally should, its a high performance web server built on top of Netty and Akka
+
+Spray has a very nice HTTP API. Few things I liked:
+ - HTTP method mapping `spray.http.HttpMethods` - `GET`, `POST` etc..
+ - `HttpRequest` and `HttpResopnse` are basically wrappers around data you would expect to have in a request and response data
+ - `spray.http.Uri` wrapper class around what you would expect to find in the URI
+ - it really is just an Akka `Actor`, with just the right amount of control exposed and the rest well hidden behind its nice APIs
+
+So, for testing/mocking all I really need is:
+
+ - HTTP method
+ - URI
+ - body(content)
+
+### Here's an implementation of the helper class
+
+```scala
+import akka.actor._
+import akka.io.IO
+import java.util.concurrent.atomic.AtomicInteger
+import spray.can.Http
+import spray.http.{Uri, HttpMethod, HttpResponse, HttpRequest}
+
+object Helpers {
+
+ lazy implicit val testSystem = ActorSystem("helpers-test-system")
+ // used to track the last assigned port
+ val lastPort = new AtomicInteger(20000)
+ // a single expected triplet of method+uri+body
+ type Replay = (HttpMethod,Uri.Path,String)
+
+ // factory method to get a replater to play around with
+ def webReplayer(replay : List[Replay]) : (ActorRef,Int) = {
+ // everytime the factory is used a new port is assigned,
+ // so many instances can co-exist and tests can run in parallel without getting
+ // on eachother's toes
+ val newPort = lastPort.addAndGet(1)
+ // instantiate a new Spray server. Yes, its that simple!
+ val newServer = testSystem.actorOf(Props(new Helpers.ReplayerActor(replay)))
+ IO(Http) ! Http.Bind(newServer, interface = "127.0.0.1", port = newPort)
+ // return a pair of the server instance and the assigned port
+ // the newPort should be used by client code to construct the remote URI with
+ (newServer,newPort)
+ }
+
+ // object to use to tell the server to shutdown..
+ object ShutTestServer
+
+ // a concrete implementation for a Spray Actor to listen to incoming HttpRequest(..) messages
+ // provided a list of messages to be matched against arrived and content to be sent back
+ class ReplayerActor(var replay :List[Replay]) extends Actor {
+ var ioServer : ActorRef = null
+ override def receive: Receive = {
+ case _ : Http.Connected =>
+ sender ! Http.Register(self)
+ ioServer = sender()
+ // this is basically a "catch all" HttpRequest pattern
+ case HttpRequest(method, uri,_,_,_) =>
+ // a test to see that this is indeed expected to happen at this point
+ if (!replay.isEmpty && replay.head._1 == method && uri.path == replay.head._2){
+ sender ! HttpResponse(entity = replay.head._3)
+ replay = replay.tail
+ } else {
+ sender ! HttpResponse(status = 400, entity = "was expecting a different request at this point")
+ }
+ case ShutTestServer => ioServer ! PoisonPill
+ case _ => //ignore
+ }
+ }
+}
+```
+
+### And now a cocrete client code example to use this tool
+
+```scala
+// some random content I would like to receive in this scenario (or test..)
+val content =
+s"""
+ |
+ |
+ |sometitle
+ |
+ |some body
+ |
+""".stripMargin
+
+// we use the factory method to get an instance of "replayer" to play around with
+val (_testWebServer,port) = Helpers.webReplayer(
+ List(
+ (GET,Uri.Path("/somesite"),content)
+ )
+)
+
+// and where ever in your actual code, use the port we got back above and have it "hit" the server
+val whatEver = YourHTTPGetter.fetch(s"http://127.0.0.1:$port/somesite")
+assert(whatEver == content)
+```
+Profit!
diff --git a/source/_posts/bootstrapping-a-new-project.md b/source/_posts/bootstrapping-a-new-project.md
new file mode 100644
index 0000000..c861414
--- /dev/null
+++ b/source/_posts/bootstrapping-a-new-project.md
@@ -0,0 +1,53 @@
+---
+title: "Bootstrapping A New Project"
+categories: ["design"]
+tags: []
+date : 2012/02/07
+---
+
+### My take on: How to give your new project a good starting point
+
+So, you have a new shiny project on your lap, so now what?
+Today everyone is talking about "agile", and how its better then the old "waterfall" methodology of doing things
+
+#### Waterfall
+The "Waterfall" approach is notorious in the software community, some highlights of why this is the case:
+
+ * It assumes that the initial requirements will not change
+ * There is only one version delivered to the stakeholders, at the end of the process
+ * All the planning and "thinking" happen only in the beginning of the project
+
+#### Agile
+Everyone likes "Agile", its the "cool" way to manage a project, it asks for less paperwork, less work upfront, built from the ground up to support changes, to name a few conceptions about it.
+While I call it "Agile" here, in reality it is a world of sub cultures, some of which are: "XP" (eXtreme Programming), "scrum", "Kanban" to name a few.
+The difference between specific cultures is not always clear, we don't necessarily have to stick to one, we can mix and match, like I am doing here, thus from this point onward I will just ignorantly call it "Agile" with out specifying the specific methodology.
+The main strengths going for the Agile methodology are
+
+ * Short upfront work phase (requirements, design ..)
+ * Iterations - project is broken down to deliverables, which is translated to real value delivered very early in the project lifetime
+ * Changes are welcome and expected and are introduced into following iterations (bugs found in previous deliverables, design "inadequacies")
+
+Doing things in small pieces is great, just ask a WEB developer, how there's nothing quite like writing a bit of code, refreshing the page and seeing the changes reflected, especially when working on a big complex project. This gives you a feeling of progress and achievement, even though the end goal is far far away..
+
+Some Agile approaches talk beyond writing business code and going into proactive bug searching using TDD (Test Driven Development), which turns out to be a good fit to the overall process as it secures quality of delivered pieces of logic.
+BDD (Behavior Driven Development) is very similar to TDD, but instead of going after "test coverage" it takes on a higher level, business specification driven approach, where the business embodied user stories are part of the actual test suite.
+
+#### My recipe for success
+
+ * User Stories
+ * Mock-ups (in the case of an app that has UI)
+ * HLD (High Level -system component- Design)
+ * Process Sequence diagram / Workflow diagram (possibly atop HLD)
+
+#### Tooling
+
+Tools that keep me productive:
+
+ * Google Document (User Stories)
+ * Google Drawing (mockups, diagrams, HLD)
+ * Lucidchart (diagrams)
+
+In addition, I use a light weight web based project management tool as a central place to keep reference to important docs and user stories, and see my progress (and others working on the project). A great tool for the job is [Trello.com](http://www.trello.com) - which free and super simple!
+
+#### REATED UPDATES:
+[User Stories](/code/2012/02/25/user-stories/)
diff --git a/source/_posts/distributed-scheduled-queue-with-redis.md b/source/_posts/distributed-scheduled-queue-with-redis.md
new file mode 100644
index 0000000..635fa0e
--- /dev/null
+++ b/source/_posts/distributed-scheduled-queue-with-redis.md
@@ -0,0 +1,86 @@
+---
+layout: post
+title: "Distributed Scheduled Queue With Redis"
+description: ""
+category:
+tags: ["redis","queue","distributed"]
+date: 2012/10/20
+---
+
+Unlike the classic FIFO queue, jobs are scheduled for execution in the future, possibly on a specific day, week, month year.. similar to "cron" in a Unix OS. Unlike the normal Unix "cron" implementation, we want it to be distributed, so that several machines(workers) can query and then execute these jobs.
+
+### Implementing
+
+ * _Planning_ - The standart Unix cron format is very flexible and has been time tested for describing schedules
+ * _Managing/Coordinating_ - The queues are stored in Redis Sorted Sets, Lua will be used for atomic execution
+ * _Execution_ - Getting the pieces together
+
+#### Planning
+
+"Unix Time" format, is the number of milliseconds since (*UTC*) 1 January 1970
+Example (* the actual unix time number is in UTC, while the formatted date is localized)
+
+ 1350745504313 == Sat Oct 20 2012 17:05:04 GMT+0200 (IST)
+
+Using this format will make it possible to slice the queue with specific point in time in mind.
+
+#### Managing/Coordinating
+
+Redis'st will be holding the job IDs as members scored by their corresponding execution times(in unix time format).
+
+
+
+Redis commands that will come in handy
+
+ * Scheduling job run: *ZADD cron:queue 1350745504313 1000*
+ * Finding a job to run: *ZRANGEBYSCORE cron:queue 1350745504314 LIMIT 0 1*
+
+#### Execution
+
+Without going into any platform specific implementation details, this is what the process looks like
+
+1. When a new schedule is added, jobs are created for upcoming execution times, which are calculated for the next two hours and added to our cron queue
+2. A special job is added to be run every hour, it will run over all the schedules, calculate and add their execution time to the scheduled queue, two hours into the future
+3. Workers will be polling the database every second to find jobs
+4. Once an entry comes up, it is removed from the queue and added to another Sorted Set queue, so we dont "forget" any jobs..
+
+As for items 3 and 4 above, it is not possible to implement this logic in the client, because for example when we find a job that needs to be executed, another worker might grab it, or worse they both "grabbed" it and are now working on the same job. To work around this, we need to write a little Lua script, since Lua scripts are executed serially, just like any other Redis command
+
+```lua
+local currentTimeMS = KEYS[1]
+local workingQueue = "scheduler:working"
+local scheduleQueue = "schedule"
+-- for easier reading, epoch time is in milliseconds
+local second = 1000
+
+local function foundJob(jobID)
+ -- implement removing the job from all other queues
+ -- implement updating job status where the actual job object is stored
+end
+
+local function findNextJob()
+ local result
+ -- check if there are any "stale" jobs
+ result = redis.call( 'ZRANGEBYSCORE', workingQueue, 0, currentTimeMS-60*second, 'LIMIT',0,1 )
+ if result[1] ~= nil then return foundJob(result[1]) end
+ -- check the normal queue
+ result = redis.call( 'ZRANGEBYSCORE', scheduleQueue, 0, currentTimeMS, 'LIMIT',0,1 )
+ if result[1] ~= nil then return foundJob(result[1]) end
+ -- if we didnt find anything return nil
+ return nil
+end
+
+return findNextJob()
+```
+
+
+And there you have it! a "Distributed scheduled queue" implemented on top of Redis.
+
+If you are going to implement this with node, I created a little cron format parser you can find here:
+[JSCron](https://github.com/romansky/JsCron)
+
+
+
+
+
+
diff --git a/source/_posts/hld-high-level-design.md b/source/_posts/hld-high-level-design.md
new file mode 100644
index 0000000..d1edf2f
--- /dev/null
+++ b/source/_posts/hld-high-level-design.md
@@ -0,0 +1,52 @@
+---
+title: "HLD High Level Design"
+categories: ['design']
+tags: ["design"]
+date: 2012/03/07
+---
+
+
+This post is a follow-up on ["Bootstrapping A New Project"](/code/2012/02/07/bootstrapping-a-new-project/)
+
+#### Previously...
+
+I've written about ["User Stories"](/code/2012/02/25/user-stories/), a tool that helps with expectation setting when working with (mostly) non-technical stakeholders.
+
+#### Designing Toward
+
+ - **database** (schema, tables, indexes..)
+ - **application layout**
+ - **deployment**
+ - **modules**
+ - **internal and external APIs**
+ - **application infra** (frameworks, tools etc..)
+
+#### Design Tools At Your Disposal
+
+ - **system architecture** ([example-Lucidchart](http://www.lucidchart.com/publicSegments/view/4f6e3aaf-0798-4bce-97ca-386a0a5a8951/image.png)) - big pieces that make up the system
+ - **activity diagram** ([more info](http://www.agilemodeling.com/artifacts/activityDiagram.htm)) - communication between components in the system
+ - **sequence diagram** ([example-Lucidchart](http://www.lucidchart.com/publicSegments/view/4f6e3ae9-8544-46f2-a469-1daf0ac9c29f/image.png), [more info](http://www.agilemodeling.com/artifacts/sequenceDiagram.htm)) - algorithms / processes high level implementation
+ - **state diagram** ([more info](http://www.agilemodeling.com/artifacts/stateMachineDiagram.htm)) - state minded processes / algorithms
+ - **mock ups / wire frames** ([example-Google Draw](https://docs.google.com/drawings/pub?id=1y-GFJZeIcqSIk2hFcf9SWjWnpemvaBU-ffdPjv1u4hs&w=960&h=720)) - for projects with GUIs
+
+#### Audiences
+
+ - **System Architect** - mostly interested in the software stack, deployment and the database design
+ - **Team Members** - mostly interested in the more lower level stuff like modules and API's
+ - **Other Teams** - mostly interested in API's
+
+Feedback on design is great for everyone involved, setup meetings with relevant parties to do DR (design review)
+
+#### UML and Other Vegetables
+
+UML is complex, you don't need to know it perfectly, start with what makes sense, and slowly pick up tutorials. Remember, its a visual tool that helps you communicate, not everyone who will look at your designs has the time (or cares for) to read throughly a book about UML and its specifics. In Einstein's words "Make everything as simple as possible, but not simpler."
+
+#### Disclosure
+
+Design does *not* last. Nobody is going to update the design as the system grows and evolves. Having said that, can still be used a good reference.
+
+
+
+
+
+
diff --git a/source/_posts/making-the-case-for-kotlin.md b/source/_posts/making-the-case-for-kotlin.md
new file mode 100644
index 0000000..4535091
--- /dev/null
+++ b/source/_posts/making-the-case-for-kotlin.md
@@ -0,0 +1,189 @@
+---
+title: "Making The Case For Kotlin"
+categories: ["code"]
+tags: ["kotlin","scala"]
+date : 2012/11/14
+---
+
+The Kotlin project page has this to say about Kotlin: "is a statically typed programming language that compiles to JVM byte codes and JavaScript." [project page](http://kotlin.jetbrains.org/)
+
+Kotlin is a very interesting project, I wanted to check it out, so I decided to write this blog post exploring some design aspects behind Kotlin, and specifically how it compares to the leading JVM "hacker" language - Scala.
+
+## Who Is Behind Kotlin?
+
+Even though there are several decent free IDE's available today (Eclipse, NetBeans, Visual Studio Express..), it (could be) surprising that JetBrains is able to rack in between 199$ for a personal license and up to 699$ for a commercial license (for companies) including 1 year update subscription (I have indeed bought one personal license for my development needs). If you are developing on top of VisualStudio you are (most) probably familiar with the 250$+ plugin ReSharper.
+
+I would argue that this makes them a good candidate, with insight into good and bad language features, but lets not get ahead of ourselves and set the expectations too high.
+
+## Why Another JVM Language?
+
+JVM is a well known platform, its weaknesses and scalability strategies are available and documented well on the web, its being used as classic Java run-time VM and for some other popular languages like Scala, Groovy, Clojure and others. Many modules have been already written in Java, normally JVM run languages are compatible with the existing Java ecosystem.
+
+## Why Static Typing?
+
+In the old (and still ongoing) battle of dynamic and static languages, the consensus is that dynamic languages are faster and easier way to get things done, on the static languages side they allow programmers harness the power of intelligent IDEs to scale up and maintain their code. Type-inferred languages try to find a middle ground approach so that you don't need to write so much boilerplate code (cue JAVA) to support the static typing, while having the benefits of a full compile time type enforcement.
+
+## Finally, Kotlin Vs. Scala - Fight!
+
+Scala today is the fastest growing JVM language and is probably competing for the same crowd as Kotlin, Kotlin put up a comparison page [link](http://confluence.jetbrains.net/display/Kotlin/Comparison+to+Scala) so we will simply go over the omissions part and try to make sense of it.
+
+### OMITTED - Implicit conversions, parameters
+
+*What is it?* Scala implicits is (IMHO) a single biggest weakness in the language, once they are used, along with the ability to boast symbols for syntax, it quickly stops being human readable and more like a DSL. Which requires many trips to the docs or even worse - the source code.
+
+`SBT` (Scala Build Tool) is the Maven/Ant of the Scala world [link](http://www.scala-sbt.org/release/docs/Examples/Full-Configuration-Example.html), its very difficult to get things done with it without many trips to google/documentation
+
+```scala
+lazy val pricing = Project (
+ "pricing",
+ file ("cdap2-pricing"),
+ settings = buildSettings ++ Seq (libraryDependencies ++= pricingDeps)
+ ) dependsOn (common, compact, server)
+```
+
+`Lift` Is a popular web framework, here's a snippet I took from one of its documentation pages [link](http://cookbook.liftweb.net/Plain+old+form+processing.html) (see the `"#result *" #> x` part)
+
+```scala
+def render = inputParam match {
+ case Full(x) =>
+ println("Input is: "+x)
+ "#result *" #> x
+
+ case _ =>
+ println("No input present! Rendering input form HTML")
+ PassThru
+}
+```
+
+
+*Thoughts - * Indeed for many people this is the part that makes Kotlin a better "scalable" language, by not allowing programmers and libraries hinder the code unreadable without an IDE
+
+### OMITTED - Overridable Type Members
+
+*What is it?* Type members in Scala allow you to manage local generic types by assign a type to a class/abstract class/trait member
+
+```scala
+abstract class HasTypedMember {
+ type typeMe
+}
+```
+
+An extending class will provide a specific type for this type type member
+
+```scala
+class ExtendingClass extends HasTypedMember {
+ type typeMe = Int
+}
+```
+
+*Thoughts - * Frankly, its not really clear why Kotlin decided to omit having typed members, I could be missing something.. Never the less Kotlin usually takes the minimalistic and simplistic route, my guess is that if this is indeed useful, the door is open to add it later..
+
+### OMITTED - Path-dependent types
+
+*What is it?* Pointing to a class under another class in Java would mean a specific type (Parent.Child), well not so in Scala. Scala makes the nested class of two same class instances different, much like you would expect two members of two different instances of a single class. Confused? here is some code to explain this better
+
+Java:
+
+```java
+public class Outer {
+ public class Inner {}
+ public void useInner(Outer.Inner inner){...}
+}
+//.. somewhere in the code
+o1 = new Outer()
+o2 = new Outer()
+i1 = new o1.inner()
+i2 = new o2.inner()
+// this would work fine
+o1.useInner(i2) // works
+o2.useInner(i1) // works
+```
+
+Scala:
+```scala
+class Outer {
+ class Inner {}
+ def useInner(inner : Inner){...}
+}
+//.. somewhere in the code..
+val o1 = new Outer
+val o2 = new Outer
+val i1 = new o1.Inner
+val i2 = new o2.Inner
+o1.useInner(i1) // works fine
+o2.useInner(i2) // works fine
+o1.useInner(i2) // THROWS type mismatch exception!
+```
+
+*Thoughts - * It makes some sense that the path to a type that can only exist in an instance is enforced as such.. It would be interesting to know why Kotlin decided to implement this feature in a similar way to Java (conservative?)
+
+### OMITTED - Existential types
+
+*What is it?* This is yet another trade-off for Scala to be able to inter-op with Java
+Let's see an example:
+
+```scala
+// Create a case class with generic type
+case class Z[T]( blah:T )
+// create an instance and cast it to a type created with existential type annotation
+Seq( Z[Int]( 5 ), Z[String]("blah") ) : Seq[Z[X] forSome { type X >: String with Int }]
+res1 = Seq[Z[_ >: String with Int]] = List(Z(5), Z(blah))
+```
+
+*Thoughts - * Scala supports type variance to describe generic type bounds, Scala designer (Odersky) chose to add existential types support due to three main reasons [link](http://www.artima.com/scalazine/articles/scalas_type_system.html): erasure, raw types (lack of generics) and Java's "wildcard" types.
+
+Kotlin took the more "purist" approach (also known as reified types) of dropping support for erasure, raw types, wildcard types and finally existential types.
+
+### OMITTED - Complicated logic for initialization of traits
+
+*What is it?* Traits are similar to abstract classes, except that you may use several traits on the same class at the same time (like interfaces).
+
+In Scala, traits are constructed before the extended classes
+
+```scala
+trait Averager {
+ val first : Int
+ val second : Int
+ val avg : Int = first / second
+}
+// if we were to naively construct a class like so:
+class Naive extends Averager {
+ val first = 20
+ val second = 10
+}
+// create instance
+> new Naive()
+java.lang.ArithmeticException: / by zero
+// what happened was that our val "avg" was computed at construction time before seeing the values we put in Naive
+```
+
+Alternatively the trait can be used with a pre-initialized anonymous class `val res = new { val first = 20 ; val second = 10 } with Averager`, also the val which depends on other vals (avg in our case) can be made "lazy" with the (obvious) lazy keyword `lazy val avg : Int = first / second`.
+
+As a side note, using def for "avg" could work, but it would mean evaluating whatever is defined under "avg" every-time this method is called.
+
+*Thoughts - * Traits in Scala can have state which could be made tricky. If you read this blog post [link](http://blog.jetbrains.com/kotlin/2011/08/multiple-inheritance-part-2-possible-directions/) you will see Kotlin developers thoughts and a nice insight into what went into designing the inheritance in Kotlin. (don't forget to read the comments!)
+
+### *OMITTED* - Custom symbolic operations
+
+*What is it?* Scala does not have a built in handling of symbols (like =,+,/ etc), they are actually method calls, as you can have symbols used for method names.
+
+*Thoughts - * This may sound useful at first, but can and (arguably) will create very confusing looking code, like we saw before in the `Lift` and `SBT` examples.
+
+### *OMITTED* - Built-in XML
+
+*What is it?* Exactly as it sounds, Scala built in XML native support, so you can do stuff like this
+
+```scala
+val holdsXml = some text
+```
+
+*Thoughts - * Having a built in support for XML as a language feature is interesting, and sometimes useful. An interview [link](http://www.techworld.com.au/article/397692/a-z_programming_languages_from_pizza_scala/) with Odersky reveals that he would consider removing the native XML support in Scala if had written Scala now.
+
+
+## Bottom Line
+
+Kotlin is an idiomatic language that draws from the many good features in todays leading languages, specifically from Scala which is a leading language on the JVM platform today. We've looked into omissions on Kotlin's part versus Scala
+
+I've very much enjoyed writing this blog post and learning about the evolution that the design of Java/Scala/Kotlin languages, and the choices the respective authors made along the way.
+
+To get a better feeling of Kotlin I might pick it up for a small project, at which point I will surely blog about it some more..
diff --git a/source/_posts/setting-up-intellij-for-android-and-scala.md b/source/_posts/setting-up-intellij-for-android-and-scala.md
new file mode 100644
index 0000000..46b8fb9
--- /dev/null
+++ b/source/_posts/setting-up-intellij-for-android-and-scala.md
@@ -0,0 +1,65 @@
+---
+title: "Setting Up IntelliJ For Android And Scala On Ubuntu"
+categories: ["code"]
+tags: ["intellij","android","scala"]
+date: 2012/02/10
+---
+
+### Prerequisites
+
+#### Android SDK
+
+ 1. get the latest [SDK](https://developer.android.com/sdk/index.html)
+
+ 2. after extracting, run `./tools/android`
+
+ 3. from the list select the Android version your afer, right now its **Android 4.0.3 (API 15)** (I also installed Android 2.3.3), click on **Install packages** to get the installation started
+
+ 4. setup your env ``echo "export ANDROID_HOME=~/Applications/android-sdk-linux/" >> ~/.bashrc``
+
+ 5. refresh your currect session `source ~/.bashrc`
+
+#### Adding SBT support to IntelliJ
+
+ 1. open **File** > **Settings** > **Plugin**
+
+ 2. click on **Browser repositories**
+
+ 3. find **SBT**, right click and select **Download and install**
+
+#### Setup [SBT](https://github.com/harrah/xsbt/wiki/Getting-Started-Setup)
+
+ 1. create folder `~/bin/` (if you dont have `~/bin` in your PATH, then you can do this: `echo "PATH=$PATH:~/bin" >> ~/.bashrc` (refresh your currect session `source ~/.bashrc`)
+
+ 2. [download](https://github.com/harrah/xsbt/wiki/Getting-Started-Setup) `sbt-launcher.jar` and place it under `~/bin`
+
+ 3. create launcher: `touch ~/bin/sbt` and then ``echo 'java -Xmx512M -jar `dirname $0`/sbt-launch.jar "$@"' >> ~/bin/sbt``
+
+ 4. make it executable: `chmod u+x ~/bin/sbt`
+
+#### Setup [android-plugin](https://github.com/jberkel/android-plugin/wiki/Getting-started) for Scala support
+
+ 1. install giter8 `curl https://raw.github.com/n8han/conscript/master/setup.sh | sh`
+
+ 2. run it `~/bin/cs n8han/giter8`
+
+#### Setup [sbt-idea](https://github.com/mpeltonen/sbt-idea)
+
+ 1. create folder (if it doesnt exist) `mkdir -p ~/.sbt/plugins/`
+
+ 2. create the file `~/.sbt/plugins/build.sbt` and add the following lines to it:
+
+ resolvers += "sbt-idea-repo" at "http://mpeltonen.github.com/maven/"
+ addSbtPlugin("com.github.mpeltonen" % "sbt-idea" % "1.0.0")
+
+### Finally, now that you have all the tools you need, proceed to creating a new project
+
+ 1. run the android plugin project setup tool: `~/bin/g8 jberkel/android-app`, follow the onscreen questions (or press return for defaults)
+ this will create the project folder and create the files needed to build an android project
+
+ 2. open the new folder `cd `
+
+ 3. setup IntelliJ project support `sbt gen-idea`
+
+ 4. open the project in IntelliJ
+
diff --git a/source/_posts/typing-in-javascript.md b/source/_posts/typing-in-javascript.md
new file mode 100644
index 0000000..c2d3fbc
--- /dev/null
+++ b/source/_posts/typing-in-javascript.md
@@ -0,0 +1,165 @@
+---
+title: "Typing in JavaScript?"
+categories : ['code']
+tags: ["nodejs","javascript"]
+date: 2012/09/22
+---
+
+"Strong typing is for people with weak memories" - not sure who originally said that, but you have got to love the enthusiasm people have for dynamic languges. What is there not to love? powerful basic language constructs such as closure, prototypal inheritance, objects as first class citizens and anonymous functions, mainly, with which you can simulate almost anything static languages have to offer!
+
+Proof you say? no problem! here are few implementations that are worthy of noting:
+
+**Compile time type checking** https://developers.google.com/closure/compiler/docs/js-for-compiler#types
+
+**Classic OO inheritance** http://mootools.net/docs/core/Class/Class & http://coffeescript.org/#classes
+
+(if you have other interesting examples let me know!)
+
+### Convention over configuration
+
+The combination of CoffeeScript's "classical" Class like system and Backbone.JS's MVC style structure, give (IMHO) the best value in terms of complexity and pure fun writing apps. The challenges I come across while building large complex applications is the inherited nature of a true dynamic language "type_lessness_".
+
+The ugly monster, that is -a large application written in JavaScript, pushed me into struggling with these two main challenges:
+
+ * Many-to-many PubSub communication
+ * 3rd party API's
+
+### Many-to-many PubSub communication
+
+While the PubSub (also known as observer pattern) in JavaScript is a powerful tool to decouple your code, having multiple callers and listeners on the same event makes it really difficult to keep track of the actual common API for these events. Lats try to reflect on this while looking at an actual implementation.
+
+(please excuse my CoffeeScript)
+
+
+**Pub/Sub implementation in Backbone/Underscore.JS**
+
+```coffee
+pubsub = _.extend {}, Backbone.Events
+pubsub.on 'myEvent', (param1, param2)->
+ x1 = param1 + param2
+ # do something with x1
+
+# in some other obscure location in the code
+pubsub.on 'myEvent', (param1)->
+ x2 = param1
+ # do something with x2
+
+# ...
+pubsub.trigger 'myEvent', 5
+# ...
+pubsub.trigger 'myEvent', 5,10
+```
+
+The "pubsub.on(event name, callback)" API is very common amongst different JS pub/sub frameworks. here's an example of how a generic implementation might look like:
+
+
+```coffee
+pubsub = {
+ _listeners : {}
+ on : (channel, callback)->
+ @_listeners[ channel ] ?= []
+ @_listeners[ channel ].push callback
+
+ trigger : (channel, params...)->
+ @_listeners[ channel ]?.forEach (cb)-> cb.apply(null, params)
+}
+
+pubsub.on 'myEvent', (param1, param2)->
+ x1 = param1 + param2
+ # ... do something with x1..
+
+# in some other obscure location in the code
+pubsub.on 'myEvent', (param1)->
+ x2 = param1
+ # .... do something with x2
+
+# ...
+pubsub.trigger 'myEvent', 5
+# ...
+pubsub.trigger 'myEvent', 5,10
+```
+
+
+If your using the very popular Socket.IO library, for realtime client/server communication, you are using the exact same API, in which case it makes it even more complicated to keep track of, since the ambiguity is manifested on both the client and the server.
+
+### 3rd party API's
+
+This is how a type related bug might look like in a dynamic language
+
+```coffee
+a = "42" # returned from some API
+a == 42
+=> false
+```
+
+## Enter "Contractor" & "Backbone-typed"
+
+Contractor
+https://github.com/romansky/Contractor
+
+Backbone-typed
+https://github.com/romansky/backbone-typed
+
+I'm introducing a convention here, that helps keep the API consistent across the application, the approach here is to create wrapper functions, that are used as both the API documentation and run-time validation facility for required and optional arguments.
+
+Example:
+
+```coffee
+# declare the events and their APIs
+Messages = {
+ LoginEvent : Contractor.Create( "LoginEvent", Contractor.Required("user ID"), Contractor.Optional("additional info") )
+ AppErrorEvent : Contractor.Create( "AppErrorEvent", Contractor.Required("user ID"), Contractor.Required("error description"), Contractor.Optional("exception") )
+}
+
+# register on event API
+pubsub.on Messages.LoginEvent, (userID, info)-> #... do stuff here
+pubsub.on Messages.AppErrorEvent, (userID, description, exception)-> #... do stuff here
+
+# trigger the event
+pubsub.apply(pubsub, Messages.LoginEvent(112233, "mobile"))
+pubsub.apply(pubsub, Messages.LoginEvent(112233))
+
+pubsub.apply(pubsub, Messages.AppErrorEvent(112233, "user did not have access to resource", e))
+pubsub.apply(pubsub, Messages.AppErrorEvent(112233)) # since we did not provide the second required argument, this will log an error and return null
+```
+
+With NodeJS, I am using the same code on the front and back end, which makes it really easy to manage my Socket.IO communication.
+
+If you have been paying attention, there's still the type issue. Since I work allot with Backbone, it made a lot of sense to write "backbone-typed" which adds optional typing to Backbone-Models.
+Example:
+
+```coffee
+Wearing = {
+ "bracelet" : "bracelet"
+ "watch" : "watch"
+}
+
+class User extends TypedModel
+ defaults: {
+ name: null
+ email: null
+ lotteryNumber: null
+ isAwesome: null
+ }
+
+ types: {
+ name: Types.String
+ email: Types.String
+ lotteryNumber: Types.Integer
+ isAwesome: Types.Boolean
+ wearing : Types.Enum(Wearing)
+ }
+
+user1 = new User({name: "foo", email: "foo@bar.com", lotteryNumber: 12345, isAwesome: true, wearing: Wearing.watch})
+user1.toJSON() #=> {name: "foo", email: "foo@bar.com", lotteryNumber: 12345, isAwesome: true, wearing: "watch"} - nothing special going on here..
+
+user2 = new User({name: "foo", email: "foo@bar.com", lotteryNumber: "54321", isAwesome: "false", wearing: "thong"})
+user2.toJSON() #=> {name: "bar", email: "bar@foo.com", lotteryNumber: 54321, isAwesome: false, wearing: null} - shit happens!
+
+user2.set({wearing: Wearing.bracelet})
+
+if user2.get("lotteryNumber") == 54321 and user2.get("wearing") == Wearing.bracelet then user2.set({isAwesome: "true"})
+user2.toJSON() #=> {name: "bar", email: "bar@foo.com", lotteryNumber: 54321, isAwesome: true, wearing: "bracelet"} - awesome for sure..
+```
+
+If you want to know a little more about how these work, please visit the respective Git repositories, and let me know what you think!
diff --git a/source/_posts/user-stories.md b/source/_posts/user-stories.md
new file mode 100644
index 0000000..f2e8301
--- /dev/null
+++ b/source/_posts/user-stories.md
@@ -0,0 +1,54 @@
+---
+title: "User Stories"
+categories: ["design"]
+tags: ["design"]
+date: 2012/02/25
+---
+
+This post is a follow-up on ["Bootstrapping A New Project"](/code/2012/02/07/bootstrapping-a-new-project/)
+
+"User Story" is an important communication tool, it is used between you (the developer) and who ever is making the requirements (the stakeholder).
+A "contract" which (unlike one written by a lawyer) is clear, understood and agreed upon.
+On our side (development) user stories embody a work unit that can be further broken down and estimated.
+
+Key value points:
+
+ * Language - its key that user stories are written in a language everyone can understand (stakeholders, developers etc..)
+ * Perspective - user stories need to be written from the perspective of the "feature user", so they represent real world value
+ * Estimation - to help you get better and more precise estimation for the project overall
+ * Prioritization - what needs to be delivered soon/later
+
+#### Lets imagine we just got a call from the product manager, where he is telling us about this new survey tool:
+
+ 1. It needs to collect answers from users via a link sent to our users by e-mail
+ 2. It needs to be summarized and sent to ... once a month on the 1st
+ 3. Managers need to be able to see past surveys
+
+Hmm.. OK, thats pretty clear, but suppose this can be written clearer, lats try using the following template: (not my invention;))
+``As a [role], I want [Requirement / Feature], So that [Justigication/ Reason]``
+
+The requirements now look like this:
+
+ 1. As an end user, I will be able to follow a link I got via e-mail, so that I can fill a survey and express my opinion
+ 2. As a manager, I will receive an e-mail with a digest of surveys, so that I can evaluate performance
+ 3. As a manager, I will be able to access old surveys, so that I can compare and evaluate performance over time
+
+By reading the above I can clearly understand the required functionality and the **motivation** behind writing it, this gives me a good **starting point**.
+Since I am the one to implement this, I will need to provide estimates, so that product can plan ahead and communicate the product road map onward.
+Unfortunately, "The Devil is In The Details" as they say, so giving estimates based on points 1,2,3 is still difficult.
+Points 2 & 3 are "huge" stories, also known as "epics", we need to further break down these stories, while still using the same language and practices:
+
+ 1. As an end user, I will receive an email with a link as a result of some communication with a company, so that I can follow it and fill a survey
+ 2. As an end user, I get a set of questions to answer, so that my opinion is collected
+ 3. As a manager, I will receive an e-mail with a digest of surveys, so that I can evaluate companies performance
+ 4. As a manager, I will have a survey management portal, so that I can overview previous results and compare to
+ 5. As a manager, I can select a specific survey, so that I can extract meaningful actionable information
+
+The functionality expected of deliverables from the points above is clear, and now I feel more comfortable estimating the time it will take me to complete each.
+Along with the estimation, these can now be prioritized against each other and against other ongoing projects..
+
+### Summery
+The beginning of a new project is the perfect time to introduce user stories, it will help with getting the conversation going, further clarifying the scope and requirements.
+While the project scope might change, user stories are (optimally) modular, decoupled and encapsulated, this should help with managing changes and their effect on the overall project.
+
+
diff --git a/source/_posts/using-contractor-with-child-processes.md b/source/_posts/using-contractor-with-child-processes.md
new file mode 100644
index 0000000..5062199
--- /dev/null
+++ b/source/_posts/using-contractor-with-child-processes.md
@@ -0,0 +1,99 @@
+---
+title: "Using Contractor With Child Processes"
+category: ["code"]
+tags: ["nodejs"]
+date: 2012/10/06
+---
+
+`Contractor` is a factory of contracts, that helps with documenting our APIs.
+The child_process facility built into node is using `EventEmitter` (v1), and does not support callbacks, so it is a great fit for `Contractor`!
+Since I needed it for a distributed worker framework I am working on, I added a helper facility called "Lawyer", it basically reads the contracts and routes them to the provided object.
+
+__Let us take the parent and child example code from Node's official documentation (http://nodejs.org/api/child_process.html) and see how simple it would be to add some documentation.__
+
+Parent
+
+```javascript
+var cp = require('child_process');
+
+var n = cp.fork(__dirname + '/sub.js');
+
+n.on('message', function(m) {
+ console.log('PARENT got message:', m);
+});
+
+n.send({ hello: 'world' });
+```
+
+Child
+
+```javascript
+process.on('message', function(m) {
+ console.log('CHILD got message:', m);
+});
+
+process.send({ foo: 'bar' });
+```
+
+__Node's child_process facility provied us with a very basic means of communication, we simply call `n.send({any: "thing"})` and it automagically recieved by the forked child. Let's spice it up with a more RPC message passing style, with contracts and lawyers__
+
+contracts.js
+
+```javascript
+var Contractor = require('contractor').Contractor;
+exports.ChildPublish = {
+ "GreetingResponse" : Contractor.Create("GreetingResponse", Contractor.Required("childs greeting response")).
+ "StatusResponse" : Contractor.Create("StatusResponse", Contractor.Required("errors count"), Contractor.Required("number of completed jobs"))
+}
+
+exports.ChildSubscribe = {
+ "Greeting": Contractor.Create("Greeting", Contractor.Required("greeting word")),
+ "StatusQuery" : Contractor.Create("StatusQuery")
+}
+
+```
+
+
+parent.js
+
+```javascript
+var contracts = require('./contracts');
+var Contractor = require('contractor').Contractor;
+var Lawyer = require('contractor').Lawyer;
+var cp = require('child_process');
+
+var n = cp.fork(__dirname + '/sub.js');
+
+n.on('message', function(m) {
+ Lawyer.Read(m, {
+ "GreetingResponse" : function(childResponse){ console.log("child responded:" + childResponse) },
+ "StatusResponse" : function(errorCount, numCompletedJobs){ console.log("childs status, errors:"+errorCount+" jobs done:" + numCompletedJobs) }
+ });
+});
+n.send(contracts.ChildSubscribe.Greeting("this is your father!"));
+n.send(contracts.ChildSubscribe.StatusQuery());
+```
+
+sub.js
+
+```javascript
+var contracts = require('./contracts');
+var Contractor = require('contractor').Contractor;
+var Lawyer = require('contractor').Lawyer;
+process.on('message', function(m) {
+ Lawyer.Read(m, {
+ "Greeting" : function(parentGreeting){ process.send( contracts.ChildPublish.GreetingResponse("Hi "+parentGreeting+" can I have 10$?")) },
+ "StatusQuery" : function(){ process.send( contracts.ChildPublish.StatusResponse(0, 42) ) }
+ });
+});
+```
+
+If we were to run these
+
+```javascript
+#> node parent.js
+child responded:Hi this is your father! can I have 10$?
+childs status, errors:0 jobs done:42
+```
+
+If you found Contractor / Lawyer interesting and/or useful, I would love to hear about it!
diff --git a/tags/android/index.html b/tags/android/index.html
deleted file mode 100644
index 049e8a3..0000000
--- a/tags/android/index.html
+++ /dev/null
@@ -1,290 +0,0 @@
-
-
-
-
-
- Tag: android | Uniformlyrandom
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
from the list select the Android version your afer, right now its Android 4.0.3 (API 15) (I also installed Android 2.3.3), click on Install packages to get the installation started
-
-
setup your env echo "export ANDROID_HOME=~/Applications/android-sdk-linux/" >> ~/.bashrc
-
-
refresh your currect session source ~/.bashrc
-
-
-
Adding SBT support to IntelliJ
-
open File > Settings > Plugin
-
-
click on Browser repositories
-
-
find SBT, right click and select Download and install
create folder ~/bin/ (if you dont have ~/bin in your PATH, then you can do this: echo "PATH=$PATH:~/bin" >> ~/.bashrc (refresh your currect session source ~/.bashrc)
Finally, now that you have all the tools you need, proceed to creating a new project
-
run the android plugin project setup tool: ~/bin/g8 jberkel/android-app, follow the onscreen questions (or press return for defaults) this will create the project folder and create the files needed to build an android project
System Architect - mostly interested in the software stack, deployment and the database design
-
Team Members - mostly interested in the more lower level stuff like modules and API’s
-
Other Teams - mostly interested in API’s
-
-
Feedback on design is great for everyone involved, setup meetings with relevant parties to do DR (design review)
-
UML and Other Vegetables
UML is complex, you don’t need to know it perfectly, start with what makes sense, and slowly pick up tutorials. Remember, its a visual tool that helps you communicate, not everyone who will look at your designs has the time (or cares for) to read throughly a book about UML and its specifics. In Einstein’s words “Make everything as simple as possible, but not simpler.”
-
Disclosure
Design does not last. Nobody is going to update the design as the system grows and evolves. Having said that, can still be used a good reference.
“User Story” is an important communication tool, it is used between you (the developer) and who ever is making the requirements (the stakeholder). A “contract” which (unlike one written by a lawyer) is clear, understood and agreed upon. On our side (development) user stories embody a work unit that can be further broken down and estimated.
-
Key value points:
-
-
Language - its key that user stories are written in a language everyone can understand (stakeholders, developers etc..)
-
Perspective - user stories need to be written from the perspective of the “feature user”, so they represent real world value
-
Estimation - to help you get better and more precise estimation for the project overall
-
Prioritization - what needs to be delivered soon/later
-
-
Lets imagine we just got a call from the product manager, where he is telling us about this new survey tool:
-
It needs to collect answers from users via a link sent to our users by e-mail
-
It needs to be summarized and sent to … once a month on the 1st
-
Managers need to be able to see past surveys
-
-
Hmm.. OK, thats pretty clear, but suppose this can be written clearer, lats try using the following template: (not my invention;)) As a [role], I want [Requirement / Feature], So that [Justigication/ Reason]
-
The requirements now look like this:
-
-
As an end user, I will be able to follow a link I got via e-mail, so that I can fill a survey and express my opinion
-
As a manager, I will receive an e-mail with a digest of surveys, so that I can evaluate performance
-
As a manager, I will be able to access old surveys, so that I can compare and evaluate performance over time
-
-
By reading the above I can clearly understand the required functionality and the motivation behind writing it, this gives me a good starting point. Since I am the one to implement this, I will need to provide estimates, so that product can plan ahead and communicate the product road map onward. Unfortunately, “The Devil is In The Details” as they say, so giving estimates based on points 1,2,3 is still difficult. Points 2 & 3 are “huge” stories, also known as “epics”, we need to further break down these stories, while still using the same language and practices:
-
-
As an end user, I will receive an email with a link as a result of some communication with a company, so that I can follow it and fill a survey
-
As an end user, I get a set of questions to answer, so that my opinion is collected
-
As a manager, I will receive an e-mail with a digest of surveys, so that I can evaluate companies performance
-
As a manager, I will have a survey management portal, so that I can overview previous results and compare to
-
As a manager, I can select a specific survey, so that I can extract meaningful actionable information
-
-
The functionality expected of deliverables from the points above is clear, and now I feel more comfortable estimating the time it will take me to complete each. Along with the estimation, these can now be prioritized against each other and against other ongoing projects..
-
Summery
The beginning of a new project is the perfect time to introduce user stories, it will help with getting the conversation going, further clarifying the scope and requirements. While the project scope might change, user stories are (optimally) modular, decoupled and encapsulated, this should help with managing changes and their effect on the overall project.
Unlike the classic FIFO queue, jobs are scheduled for execution in the future, possibly on a specific day, week, month year.. similar to “cron” in a Unix OS. Unlike the normal Unix “cron” implementation, we want it to be distributed, so that several machines(workers) can query and then execute these jobs.
-
Implementing
-
Planning - The standart Unix cron format is very flexible and has been time tested for describing schedules
-
Managing/Coordinating - The queues are stored in Redis Sorted Sets, Lua will be used for atomic execution
-
Execution - Getting the pieces together
-
-
Planning
“Unix Time” format, is the number of milliseconds since (UTC) 1 January 1970 Example (* the actual unix time number is in UTC, while the formatted date is localized)
-
1350745504313 == Sat Oct 20 2012 17:05:04 GMT+0200 (IST)
-
Using this format will make it possible to slice the queue with specific point in time in mind.
-
Managing/Coordinating
Redis’st will be holding the job IDs as members scored by their corresponding execution times(in unix time format).
Finding a job to run: ZRANGEBYSCORE cron:queue 1350745504314 LIMIT 0 1
-
-
Execution
Without going into any platform specific implementation details, this is what the process looks like
-
-
When a new schedule is added, jobs are created for upcoming execution times, which are calculated for the next two hours and added to our cron queue
-
A special job is added to be run every hour, it will run over all the schedules, calculate and add their execution time to the scheduled queue, two hours into the future
-
Workers will be polling the database every second to find jobs
-
Once an entry comes up, it is removed from the queue and added to another Sorted Set queue, so we dont “forget” any jobs..
-
-
As for items 3 and 4 above, it is not possible to implement this logic in the client, because for example when we find a job that needs to be executed, another worker might grab it, or worse they both “grabbed” it and are now working on the same job. To work around this, we need to write a little Lua script, since Lua scripts are executed serially, just like any other Redis command
local currentTimeMS = KEYS[1] local workingQueue = "scheduler:working" local scheduleQueue = "schedule" -- for easier reading, epoch time is in milliseconds local second = 1000
localfunctionfoundJob(jobID) -- implement removing the job from all other queues -- implement updating job status where the actual job object is stored end
localfunctionfindNextJob() local result -- check if there are any "stale" jobs result = redis.call( 'ZRANGEBYSCORE', workingQueue, 0, currentTimeMS-60*second, 'LIMIT',0,1 ) if result[1] ~= nilthenreturn foundJob(result[1]) end -- check the normal queue result = redis.call( 'ZRANGEBYSCORE', scheduleQueue, 0, currentTimeMS, 'LIMIT',0,1 ) if result[1] ~= nilthenreturn foundJob(result[1]) end -- if we didnt find anything return nil returnnil end
return findNextJob()
-
And there you have it! a “Distributed scheduled queue” implemented on top of Redis.
-
If you are going to implement this with node, I created a little cron format parser you can find here: JSCron
from the list select the Android version your afer, right now its Android 4.0.3 (API 15) (I also installed Android 2.3.3), click on Install packages to get the installation started
-
-
setup your env echo "export ANDROID_HOME=~/Applications/android-sdk-linux/" >> ~/.bashrc
-
-
refresh your currect session source ~/.bashrc
-
-
-
Adding SBT support to IntelliJ
-
open File > Settings > Plugin
-
-
click on Browser repositories
-
-
find SBT, right click and select Download and install
create folder ~/bin/ (if you dont have ~/bin in your PATH, then you can do this: echo "PATH=$PATH:~/bin" >> ~/.bashrc (refresh your currect session source ~/.bashrc)
Finally, now that you have all the tools you need, proceed to creating a new project
-
run the android plugin project setup tool: ~/bin/g8 jberkel/android-app, follow the onscreen questions (or press return for defaults) this will create the project folder and create the files needed to build an android project
“Strong typing is for people with weak memories” - not sure who originally said that, but you have got to love the enthusiasm people have for dynamic languges. What is there not to love? powerful basic language constructs such as closure, prototypal inheritance, objects as first class citizens and anonymous functions, mainly, with which you can simulate almost anything static languages have to offer!
-
Proof you say? no problem! here are few implementations that are worthy of noting:
(if you have other interesting examples let me know!)
-
Convention over configuration
The combination of CoffeeScript’s “classical” Class like system and Backbone.JS’s MVC style structure, give (IMHO) the best value in terms of complexity and pure fun writing apps. The challenges I come across while building large complex applications is the inherited nature of a true dynamic language “typelessness“.
-
The ugly monster, that is -a large application written in JavaScript, pushed me into struggling with these two main challenges:
-
-
Many-to-many PubSub communication
-
3rd party API’s
-
-
Many-to-many PubSub communication
While the PubSub (also known as observer pattern) in JavaScript is a powerful tool to decouple your code, having multiple callers and listeners on the same event makes it really difficult to keep track of the actual common API for these events. Lats try to reflect on this while looking at an actual implementation.
-
(please excuse my CoffeeScript)
-
Pub/Sub implementation in Backbone/Underscore.JS
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14
pubsub = _.extend {}, Backbone.Events pubsub.on 'myEvent', (param1, param2)-> x1 = param1 + param2 # do something with x1
# in some other obscure location in the code pubsub.on 'myEvent', (param1)-> x2 = param1 # do something with x2
The “pubsub.on(event name, callback)” API is very common amongst different JS pub/sub frameworks. here’s an example of how a generic implementation might look like:
If your using the very popular Socket.IO library, for realtime client/server communication, you are using the exact same API, in which case it makes it even more complicated to keep track of, since the ambiguity is manifested on both the client and the server.
-
3rd party API’s
This is how a type related bug might look like in a dynamic language
-
1 2 3
a = "42" # returned from some API a == 42 => false
I’m introducing a convention here, that helps keep the API consistent across the application, the approach here is to create wrapper functions, that are used as both the API documentation and run-time validation facility for required and optional arguments.
-
Example:
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
# declare the events and their APIs Messages = { LoginEvent : Contractor.Create( "LoginEvent", Contractor.Required("user ID"), Contractor.Optional("additional info") ) AppErrorEvent : Contractor.Create( "AppErrorEvent", Contractor.Required("user ID"), Contractor.Required("error description"), Contractor.Optional("exception") ) }
# register on event API pubsub.on Messages.LoginEvent, (userID, info)-> #... do stuff here pubsub.on Messages.AppErrorEvent, (userID, description, exception)-> #... do stuff here
# trigger the event pubsub.apply(pubsub, Messages.LoginEvent(112233, "mobile")) pubsub.apply(pubsub, Messages.LoginEvent(112233))
pubsub.apply(pubsub, Messages.AppErrorEvent(112233, "user did not have access to resource", e)) pubsub.apply(pubsub, Messages.AppErrorEvent(112233)) # since we did not provide the second required argument, this will log an error and return null
-
With NodeJS, I am using the same code on the front and back end, which makes it really easy to manage my Socket.IO communication.
-
If you have been paying attention, there’s still the type issue. Since I work allot with Backbone, it made a lot of sense to write “backbone-typed” which adds optional typing to Backbone-Models. Example:
The Kotlin project page has this to say about Kotlin: “is a statically typed programming language that compiles to JVM byte codes and JavaScript.” project page
-
Kotlin is a very interesting project, I wanted to check it out, so I decided to write this blog post exploring some design aspects behind Kotlin, and specifically how it compares to the leading JVM “hacker” language - Scala.
-
Who Is Behind Kotlin?
Even though there are several decent free IDE’s available today (Eclipse, NetBeans, Visual Studio Express..), it (could be) surprising that JetBrains is able to rack in between 199$ for a personal license and up to 699$ for a commercial license (for companies) including 1 year update subscription (I have indeed bought one personal license for my development needs). If you are developing on top of VisualStudio you are (most) probably familiar with the 250$+ plugin ReSharper.
-
I would argue that this makes them a good candidate, with insight into good and bad language features, but lets not get ahead of ourselves and set the expectations too high.
-
Why Another JVM Language?
JVM is a well known platform, its weaknesses and scalability strategies are available and documented well on the web, its being used as classic Java run-time VM and for some other popular languages like Scala, Groovy, Clojure and others. Many modules have been already written in Java, normally JVM run languages are compatible with the existing Java ecosystem.
-
Why Static Typing?
In the old (and still ongoing) battle of dynamic and static languages, the consensus is that dynamic languages are faster and easier way to get things done, on the static languages side they allow programmers harness the power of intelligent IDEs to scale up and maintain their code. Type-inferred languages try to find a middle ground approach so that you don’t need to write so much boilerplate code (cue JAVA) to support the static typing, while having the benefits of a full compile time type enforcement.
-
Finally, Kotlin Vs. Scala - Fight!
Scala today is the fastest growing JVM language and is probably competing for the same crowd as Kotlin, Kotlin put up a comparison page link so we will simply go over the omissions part and try to make sense of it.
-
OMITTED - Implicit conversions, parameters
What is it? Scala implicits is (IMHO) a single biggest weakness in the language, once they are used, along with the ability to boast symbols for syntax, it quickly stops being human readable and more like a DSL. Which requires many trips to the docs or even worse - the source code.
-
SBT (Scala Build Tool) is the Maven/Ant of the Scala world link, its very difficult to get things done with it without many trips to google/documentation
Lift Is a popular web framework, here’s a snippet I took from one of its documentation pages link (see the "#result *" #> x part)
-
1 2 3 4 5 6 7 8 9
defrender= inputParam match { caseFull(x) => println("Input is: "+x) "#result *" #> x case _ => println("No input present! Rendering input form HTML") PassThru }
-
Thoughts - Indeed for many people this is the part that makes Kotlin a better “scalable” language, by not allowing programmers and libraries hinder the code unreadable without an IDE
-
OMITTED - Overridable Type Members
What is it? Type members in Scala allow you to manage local generic types by assign a type to a class/abstract class/trait member
-
1 2 3
abstractclassHasTypedMember{ typetypeMe }
-
An extending class will provide a specific type for this type type member
-
1 2 3
classExtendingClassextendsHasTypedMember{ typetypeMe= Int }
-
Thoughts - Frankly, its not really clear why Kotlin decided to omit having typed members, I could be missing something.. Never the less Kotlin usually takes the minimalistic and simplistic route, my guess is that if this is indeed useful, the door is open to add it later..
-
OMITTED - Path-dependent types
What is it? Pointing to a class under another class in Java would mean a specific type (Parent.Child), well not so in Scala. Scala makes the nested class of two same class instances different, much like you would expect two members of two different instances of a single class. Confused? here is some code to explain this better
-
Java:
-
1 2 3 4 5 6 7 8 9 10 11 12
publicclassOuter{ publicclassInner{} publicvoiduseInner(Outer.Inner inner){...} } //.. somewhere in the code o1 = new Outer() o2 = new Outer() i1 = new o1.inner() i2 = new o2.inner() // this would work fine o1.useInner(i2) // works o2.useInner(i1) // works
-
Scala:
1 2 3 4 5 6 7 8 9 10 11 12
classOuter{ classInner{} defuseInner(inner : Inner){...} } //.. somewhere in the code.. val o1 = newOuter val o2 = newOuter val i1 = new o1.Inner val i2 = new o2.Inner o1.useInner(i1) // works fine o2.useInner(i2) // works fine o1.useInner(i2) // THROWS type mismatch exception!
-
Thoughts - It makes some sense that the path to a type that can only exist in an instance is enforced as such.. It would be interesting to know why Kotlin decided to implement this feature in a similar way to Java (conservative?)
-
OMITTED - Existential types
What is it? This is yet another trade-off for Scala to be able to inter-op with Java Let’s see an example:
-
1 2 3 4 5
// Create a case class with generic type caseclassZ[T]( blah:T) // create an instance and cast it to a type created with existential type annotation Seq( Z[Int]( 5 ), Z[String]("blah") ) : Seq[Z[X] forSome { typeX>: StringwithInt }] res1 = Seq[Z[_ >: StringwithInt]] = List(Z(5), Z(blah))
-
Thoughts - Scala supports type variance to describe generic type bounds, Scala designer (Odersky) chose to add existential types support due to three main reasons link: erasure, raw types (lack of generics) and Java’s “wildcard” types.
-
Kotlin took the more “purist” approach (also known as reified types) of dropping support for erasure, raw types, wildcard types and finally existential types.
-
OMITTED - Complicated logic for initialization of traits
What is it? Traits are similar to abstract classes, except that you may use several traits on the same class at the same time (like interfaces).
-
In Scala, traits are constructed before the extended classes
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14
traitAverager{ val first : Int val second : Int val avg : Int = first / second } // if we were to naively construct a class like so: classNaiveextendsAverager{ val first = 20 val second = 10 } // create instance > newNaive() java.lang.ArithmeticException: / by zero // what happened was that our val "avg" was computed at construction time before seeing the values we put in Naive
-
Alternatively the trait can be used with a pre-initialized anonymous class val res = new { val first = 20 ; val second = 10 } with Averager, also the val which depends on other vals (avg in our case) can be made “lazy” with the (obvious) lazy keyword lazy val avg : Int = first / second.
-
As a side note, using def for “avg” could work, but it would mean evaluating whatever is defined under “avg” every-time this method is called.
-
Thoughts - Traits in Scala can have state which could be made tricky. If you read this blog post link you will see Kotlin developers thoughts and a nice insight into what went into designing the inheritance in Kotlin. (don’t forget to read the comments!)
-
OMITTED - Custom symbolic operations
What is it? Scala does not have a built in handling of symbols (like =,+,/ etc), they are actually method calls, as you can have symbols used for method names.
-
Thoughts - This may sound useful at first, but can and (arguably) will create very confusing looking code, like we saw before in the Lift and SBT examples.
-
OMITTED - Built-in XML
What is it? Exactly as it sounds, Scala built in XML native support, so you can do stuff like this
-
1
val holdsXml = <outer><inner>some text</inner></outer>
-
Thoughts - Having a built in support for XML as a language feature is interesting, and sometimes useful. An interview link with Odersky reveals that he would consider removing the native XML support in Scala if had written Scala now.
-
Bottom Line
Kotlin is an idiomatic language that draws from the many good features in todays leading languages, specifically from Scala which is a leading language on the JVM platform today. We’ve looked into omissions on Kotlin’s part versus Scala
-
I’ve very much enjoyed writing this blog post and learning about the evolution that the design of Java/Scala/Kotlin languages, and the choices the respective authors made along the way.
-
To get a better feeling of Kotlin I might pick it up for a small project, at which point I will surely blog about it some more..
Contractor is a factory of contracts, that helps with documenting our APIs. The child_process facility built into node is using EventEmitter (v1), and does not support callbacks, so it is a great fit for Contractor! Since I needed it for a distributed worker framework I am working on, I added a helper facility called “Lawyer”, it basically reads the contracts and routes them to the provided object.
-
Let us take the parent and child example code from Node’s official documentation (http://nodejs.org/api/child_process.html) and see how simple it would be to add some documentation.
Node’s child_process facility provied us with a very basic means of communication, we simply call n.send({any: "thing"}) and it automagically recieved by the forked child. Let’s spice it up with a more RPC message passing style, with contracts and lawyers
var contracts = require('./contracts'); var Contractor = require('contractor').Contractor; var Lawyer = require('contractor').Lawyer; var cp = require('child_process');
“Strong typing is for people with weak memories” - not sure who originally said that, but you have got to love the enthusiasm people have for dynamic languges. What is there not to love? powerful basic language constructs such as closure, prototypal inheritance, objects as first class citizens and anonymous functions, mainly, with which you can simulate almost anything static languages have to offer!
-
Proof you say? no problem! here are few implementations that are worthy of noting:
(if you have other interesting examples let me know!)
-
Convention over configuration
The combination of CoffeeScript’s “classical” Class like system and Backbone.JS’s MVC style structure, give (IMHO) the best value in terms of complexity and pure fun writing apps. The challenges I come across while building large complex applications is the inherited nature of a true dynamic language “typelessness“.
-
The ugly monster, that is -a large application written in JavaScript, pushed me into struggling with these two main challenges:
-
-
Many-to-many PubSub communication
-
3rd party API’s
-
-
Many-to-many PubSub communication
While the PubSub (also known as observer pattern) in JavaScript is a powerful tool to decouple your code, having multiple callers and listeners on the same event makes it really difficult to keep track of the actual common API for these events. Lats try to reflect on this while looking at an actual implementation.
-
(please excuse my CoffeeScript)
-
Pub/Sub implementation in Backbone/Underscore.JS
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14
pubsub = _.extend {}, Backbone.Events pubsub.on 'myEvent', (param1, param2)-> x1 = param1 + param2 # do something with x1
# in some other obscure location in the code pubsub.on 'myEvent', (param1)-> x2 = param1 # do something with x2
The “pubsub.on(event name, callback)” API is very common amongst different JS pub/sub frameworks. here’s an example of how a generic implementation might look like:
If your using the very popular Socket.IO library, for realtime client/server communication, you are using the exact same API, in which case it makes it even more complicated to keep track of, since the ambiguity is manifested on both the client and the server.
-
3rd party API’s
This is how a type related bug might look like in a dynamic language
-
1 2 3
a = "42" # returned from some API a == 42 => false
I’m introducing a convention here, that helps keep the API consistent across the application, the approach here is to create wrapper functions, that are used as both the API documentation and run-time validation facility for required and optional arguments.
-
Example:
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
# declare the events and their APIs Messages = { LoginEvent : Contractor.Create( "LoginEvent", Contractor.Required("user ID"), Contractor.Optional("additional info") ) AppErrorEvent : Contractor.Create( "AppErrorEvent", Contractor.Required("user ID"), Contractor.Required("error description"), Contractor.Optional("exception") ) }
# register on event API pubsub.on Messages.LoginEvent, (userID, info)-> #... do stuff here pubsub.on Messages.AppErrorEvent, (userID, description, exception)-> #... do stuff here
# trigger the event pubsub.apply(pubsub, Messages.LoginEvent(112233, "mobile")) pubsub.apply(pubsub, Messages.LoginEvent(112233))
pubsub.apply(pubsub, Messages.AppErrorEvent(112233, "user did not have access to resource", e)) pubsub.apply(pubsub, Messages.AppErrorEvent(112233)) # since we did not provide the second required argument, this will log an error and return null
-
With NodeJS, I am using the same code on the front and back end, which makes it really easy to manage my Socket.IO communication.
-
If you have been paying attention, there’s still the type issue. Since I work allot with Backbone, it made a lot of sense to write “backbone-typed” which adds optional typing to Backbone-Models. Example:
Unlike the classic FIFO queue, jobs are scheduled for execution in the future, possibly on a specific day, week, month year.. similar to “cron” in a Unix OS. Unlike the normal Unix “cron” implementation, we want it to be distributed, so that several machines(workers) can query and then execute these jobs.
-
Implementing
-
Planning - The standart Unix cron format is very flexible and has been time tested for describing schedules
-
Managing/Coordinating - The queues are stored in Redis Sorted Sets, Lua will be used for atomic execution
-
Execution - Getting the pieces together
-
-
Planning
“Unix Time” format, is the number of milliseconds since (UTC) 1 January 1970 Example (* the actual unix time number is in UTC, while the formatted date is localized)
-
1350745504313 == Sat Oct 20 2012 17:05:04 GMT+0200 (IST)
-
Using this format will make it possible to slice the queue with specific point in time in mind.
-
Managing/Coordinating
Redis’st will be holding the job IDs as members scored by their corresponding execution times(in unix time format).
Finding a job to run: ZRANGEBYSCORE cron:queue 1350745504314 LIMIT 0 1
-
-
Execution
Without going into any platform specific implementation details, this is what the process looks like
-
-
When a new schedule is added, jobs are created for upcoming execution times, which are calculated for the next two hours and added to our cron queue
-
A special job is added to be run every hour, it will run over all the schedules, calculate and add their execution time to the scheduled queue, two hours into the future
-
Workers will be polling the database every second to find jobs
-
Once an entry comes up, it is removed from the queue and added to another Sorted Set queue, so we dont “forget” any jobs..
-
-
As for items 3 and 4 above, it is not possible to implement this logic in the client, because for example when we find a job that needs to be executed, another worker might grab it, or worse they both “grabbed” it and are now working on the same job. To work around this, we need to write a little Lua script, since Lua scripts are executed serially, just like any other Redis command
local currentTimeMS = KEYS[1] local workingQueue = "scheduler:working" local scheduleQueue = "schedule" -- for easier reading, epoch time is in milliseconds local second = 1000
localfunctionfoundJob(jobID) -- implement removing the job from all other queues -- implement updating job status where the actual job object is stored end
localfunctionfindNextJob() local result -- check if there are any "stale" jobs result = redis.call( 'ZRANGEBYSCORE', workingQueue, 0, currentTimeMS-60*second, 'LIMIT',0,1 ) if result[1] ~= nilthenreturn foundJob(result[1]) end -- check the normal queue result = redis.call( 'ZRANGEBYSCORE', scheduleQueue, 0, currentTimeMS, 'LIMIT',0,1 ) if result[1] ~= nilthenreturn foundJob(result[1]) end -- if we didnt find anything return nil returnnil end
return findNextJob()
-
And there you have it! a “Distributed scheduled queue” implemented on top of Redis.
-
If you are going to implement this with node, I created a little cron format parser you can find here: JSCron
Unlike the classic FIFO queue, jobs are scheduled for execution in the future, possibly on a specific day, week, month year.. similar to “cron” in a Unix OS. Unlike the normal Unix “cron” implementation, we want it to be distributed, so that several machines(workers) can query and then execute these jobs.
-
Implementing
-
Planning - The standart Unix cron format is very flexible and has been time tested for describing schedules
-
Managing/Coordinating - The queues are stored in Redis Sorted Sets, Lua will be used for atomic execution
-
Execution - Getting the pieces together
-
-
Planning
“Unix Time” format, is the number of milliseconds since (UTC) 1 January 1970 Example (* the actual unix time number is in UTC, while the formatted date is localized)
-
1350745504313 == Sat Oct 20 2012 17:05:04 GMT+0200 (IST)
-
Using this format will make it possible to slice the queue with specific point in time in mind.
-
Managing/Coordinating
Redis’st will be holding the job IDs as members scored by their corresponding execution times(in unix time format).
Finding a job to run: ZRANGEBYSCORE cron:queue 1350745504314 LIMIT 0 1
-
-
Execution
Without going into any platform specific implementation details, this is what the process looks like
-
-
When a new schedule is added, jobs are created for upcoming execution times, which are calculated for the next two hours and added to our cron queue
-
A special job is added to be run every hour, it will run over all the schedules, calculate and add their execution time to the scheduled queue, two hours into the future
-
Workers will be polling the database every second to find jobs
-
Once an entry comes up, it is removed from the queue and added to another Sorted Set queue, so we dont “forget” any jobs..
-
-
As for items 3 and 4 above, it is not possible to implement this logic in the client, because for example when we find a job that needs to be executed, another worker might grab it, or worse they both “grabbed” it and are now working on the same job. To work around this, we need to write a little Lua script, since Lua scripts are executed serially, just like any other Redis command
local currentTimeMS = KEYS[1] local workingQueue = "scheduler:working" local scheduleQueue = "schedule" -- for easier reading, epoch time is in milliseconds local second = 1000
localfunctionfoundJob(jobID) -- implement removing the job from all other queues -- implement updating job status where the actual job object is stored end
localfunctionfindNextJob() local result -- check if there are any "stale" jobs result = redis.call( 'ZRANGEBYSCORE', workingQueue, 0, currentTimeMS-60*second, 'LIMIT',0,1 ) if result[1] ~= nilthenreturn foundJob(result[1]) end -- check the normal queue result = redis.call( 'ZRANGEBYSCORE', scheduleQueue, 0, currentTimeMS, 'LIMIT',0,1 ) if result[1] ~= nilthenreturn foundJob(result[1]) end -- if we didnt find anything return nil returnnil end
return findNextJob()
-
And there you have it! a “Distributed scheduled queue” implemented on top of Redis.
-
If you are going to implement this with node, I created a little cron format parser you can find here: JSCron
If you need to test/mock 3rd party API’s / a crawler / external web service, and you dont want to use some fancy DI framework (I hate the magic in DI frameworks, I use constructor DI instead..), anyway, here’s a helper class I put togather to that end
-
If you haven’t got a chance to try out Spray, you totally should, its a high performance web server built on top of Netty and Akka
-
Spray has a very nice HTTP API. Few things I liked:
-
-
HTTP method mapping spray.http.HttpMethods - GET, POST etc..
-
HttpRequest and HttpResopnse are basically wrappers around data you would expect to have in a request and response data
-
spray.http.Uri wrapper class around what you would expect to find in the URI
-
it really is just an Akka Actor, with just the right amount of control exposed and the rest well hidden behind its nice APIs
import akka.actor._ import akka.io.IO import java.util.concurrent.atomic.AtomicInteger import spray.can.Http import spray.http.{Uri, HttpMethod, HttpResponse, HttpRequest} objectHelpers{ lazyimplicitval testSystem = ActorSystem("helpers-test-system") // used to track the last assigned port val lastPort = newAtomicInteger(20000) // a single expected triplet of method+uri+body typeReplay= (HttpMethod,Uri.Path,String) // factory method to get a replater to play around with defwebReplayer(replay : List[Replay]) : (ActorRef,Int) = { // everytime the factory is used a new port is assigned, // so many instances can co-exist and tests can run in parallel without getting // on eachother's toes val newPort = lastPort.addAndGet(1) // instantiate a new Spray server. Yes, its that simple! val newServer = testSystem.actorOf(Props(newHelpers.ReplayerActor(replay))) IO(Http) ! Http.Bind(newServer, interface = "127.0.0.1", port = newPort) // return a pair of the server instance and the assigned port // the newPort should be used by client code to construct the remote URI with (newServer,newPort) } // object to use to tell the server to shutdown.. objectShutTestServer // a concrete implementation for a Spray Actor to listen to incoming HttpRequest(..) messages // provided a list of messages to be matched against arrived and content to be sent back classReplayerActor(var replay :List[Replay]) extendsActor{ var ioServer : ActorRef = null overridedefreceive: Receive = { case _ : Http.Connected => sender ! Http.Register(self) ioServer = sender() // this is basically a "catch all" HttpRequest pattern caseHttpRequest(method, uri,_,_,_) => // a test to see that this is indeed expected to happen at this point if (!replay.isEmpty && replay.head._1 == method && uri.path == replay.head._2){ sender ! HttpResponse(entity = replay.head._3) replay = replay.tail } else { sender ! HttpResponse(status = 400, entity = "was expecting a different request at this point") } caseShutTestServer => ioServer ! PoisonPill case _ => //ignore } } }
-
And now a cocrete client code example to use this tool
// some random content I would like to receive in this scenario (or test..) val content = s""" |<html> |<head> |<title>sometitle</title> |</head> |<body>some body</body> |</html> """.stripMargin // we use the factory method to get an instance of "replayer" to play around with val (_testWebServer,port) = Helpers.webReplayer( List( (GET,Uri.Path("/somesite"),content) ) ) // and where ever in your actual code, use the port we got back above and have it "hit" the server val whatEver = YourHTTPGetter.fetch(s"http://127.0.0.1:$port/somesite") assert(whatEver == content)
The Kotlin project page has this to say about Kotlin: “is a statically typed programming language that compiles to JVM byte codes and JavaScript.” project page
-
Kotlin is a very interesting project, I wanted to check it out, so I decided to write this blog post exploring some design aspects behind Kotlin, and specifically how it compares to the leading JVM “hacker” language - Scala.
-
Who Is Behind Kotlin?
Even though there are several decent free IDE’s available today (Eclipse, NetBeans, Visual Studio Express..), it (could be) surprising that JetBrains is able to rack in between 199$ for a personal license and up to 699$ for a commercial license (for companies) including 1 year update subscription (I have indeed bought one personal license for my development needs). If you are developing on top of VisualStudio you are (most) probably familiar with the 250$+ plugin ReSharper.
-
I would argue that this makes them a good candidate, with insight into good and bad language features, but lets not get ahead of ourselves and set the expectations too high.
-
Why Another JVM Language?
JVM is a well known platform, its weaknesses and scalability strategies are available and documented well on the web, its being used as classic Java run-time VM and for some other popular languages like Scala, Groovy, Clojure and others. Many modules have been already written in Java, normally JVM run languages are compatible with the existing Java ecosystem.
-
Why Static Typing?
In the old (and still ongoing) battle of dynamic and static languages, the consensus is that dynamic languages are faster and easier way to get things done, on the static languages side they allow programmers harness the power of intelligent IDEs to scale up and maintain their code. Type-inferred languages try to find a middle ground approach so that you don’t need to write so much boilerplate code (cue JAVA) to support the static typing, while having the benefits of a full compile time type enforcement.
-
Finally, Kotlin Vs. Scala - Fight!
Scala today is the fastest growing JVM language and is probably competing for the same crowd as Kotlin, Kotlin put up a comparison page link so we will simply go over the omissions part and try to make sense of it.
-
OMITTED - Implicit conversions, parameters
What is it? Scala implicits is (IMHO) a single biggest weakness in the language, once they are used, along with the ability to boast symbols for syntax, it quickly stops being human readable and more like a DSL. Which requires many trips to the docs or even worse - the source code.
-
SBT (Scala Build Tool) is the Maven/Ant of the Scala world link, its very difficult to get things done with it without many trips to google/documentation
Lift Is a popular web framework, here’s a snippet I took from one of its documentation pages link (see the "#result *" #> x part)
-
1 2 3 4 5 6 7 8 9
defrender= inputParam match { caseFull(x) => println("Input is: "+x) "#result *" #> x case _ => println("No input present! Rendering input form HTML") PassThru }
-
Thoughts - Indeed for many people this is the part that makes Kotlin a better “scalable” language, by not allowing programmers and libraries hinder the code unreadable without an IDE
-
OMITTED - Overridable Type Members
What is it? Type members in Scala allow you to manage local generic types by assign a type to a class/abstract class/trait member
-
1 2 3
abstractclassHasTypedMember{ typetypeMe }
-
An extending class will provide a specific type for this type type member
-
1 2 3
classExtendingClassextendsHasTypedMember{ typetypeMe= Int }
-
Thoughts - Frankly, its not really clear why Kotlin decided to omit having typed members, I could be missing something.. Never the less Kotlin usually takes the minimalistic and simplistic route, my guess is that if this is indeed useful, the door is open to add it later..
-
OMITTED - Path-dependent types
What is it? Pointing to a class under another class in Java would mean a specific type (Parent.Child), well not so in Scala. Scala makes the nested class of two same class instances different, much like you would expect two members of two different instances of a single class. Confused? here is some code to explain this better
-
Java:
-
1 2 3 4 5 6 7 8 9 10 11 12
publicclassOuter{ publicclassInner{} publicvoiduseInner(Outer.Inner inner){...} } //.. somewhere in the code o1 = new Outer() o2 = new Outer() i1 = new o1.inner() i2 = new o2.inner() // this would work fine o1.useInner(i2) // works o2.useInner(i1) // works
-
Scala:
1 2 3 4 5 6 7 8 9 10 11 12
classOuter{ classInner{} defuseInner(inner : Inner){...} } //.. somewhere in the code.. val o1 = newOuter val o2 = newOuter val i1 = new o1.Inner val i2 = new o2.Inner o1.useInner(i1) // works fine o2.useInner(i2) // works fine o1.useInner(i2) // THROWS type mismatch exception!
-
Thoughts - It makes some sense that the path to a type that can only exist in an instance is enforced as such.. It would be interesting to know why Kotlin decided to implement this feature in a similar way to Java (conservative?)
-
OMITTED - Existential types
What is it? This is yet another trade-off for Scala to be able to inter-op with Java Let’s see an example:
-
1 2 3 4 5
// Create a case class with generic type caseclassZ[T]( blah:T) // create an instance and cast it to a type created with existential type annotation Seq( Z[Int]( 5 ), Z[String]("blah") ) : Seq[Z[X] forSome { typeX>: StringwithInt }] res1 = Seq[Z[_ >: StringwithInt]] = List(Z(5), Z(blah))
-
Thoughts - Scala supports type variance to describe generic type bounds, Scala designer (Odersky) chose to add existential types support due to three main reasons link: erasure, raw types (lack of generics) and Java’s “wildcard” types.
-
Kotlin took the more “purist” approach (also known as reified types) of dropping support for erasure, raw types, wildcard types and finally existential types.
-
OMITTED - Complicated logic for initialization of traits
What is it? Traits are similar to abstract classes, except that you may use several traits on the same class at the same time (like interfaces).
-
In Scala, traits are constructed before the extended classes
-
1 2 3 4 5 6 7 8 9 10 11 12 13 14
traitAverager{ val first : Int val second : Int val avg : Int = first / second } // if we were to naively construct a class like so: classNaiveextendsAverager{ val first = 20 val second = 10 } // create instance > newNaive() java.lang.ArithmeticException: / by zero // what happened was that our val "avg" was computed at construction time before seeing the values we put in Naive
-
Alternatively the trait can be used with a pre-initialized anonymous class val res = new { val first = 20 ; val second = 10 } with Averager, also the val which depends on other vals (avg in our case) can be made “lazy” with the (obvious) lazy keyword lazy val avg : Int = first / second.
-
As a side note, using def for “avg” could work, but it would mean evaluating whatever is defined under “avg” every-time this method is called.
-
Thoughts - Traits in Scala can have state which could be made tricky. If you read this blog post link you will see Kotlin developers thoughts and a nice insight into what went into designing the inheritance in Kotlin. (don’t forget to read the comments!)
-
OMITTED - Custom symbolic operations
What is it? Scala does not have a built in handling of symbols (like =,+,/ etc), they are actually method calls, as you can have symbols used for method names.
-
Thoughts - This may sound useful at first, but can and (arguably) will create very confusing looking code, like we saw before in the Lift and SBT examples.
-
OMITTED - Built-in XML
What is it? Exactly as it sounds, Scala built in XML native support, so you can do stuff like this
-
1
val holdsXml = <outer><inner>some text</inner></outer>
-
Thoughts - Having a built in support for XML as a language feature is interesting, and sometimes useful. An interview link with Odersky reveals that he would consider removing the native XML support in Scala if had written Scala now.
-
Bottom Line
Kotlin is an idiomatic language that draws from the many good features in todays leading languages, specifically from Scala which is a leading language on the JVM platform today. We’ve looked into omissions on Kotlin’s part versus Scala
-
I’ve very much enjoyed writing this blog post and learning about the evolution that the design of Java/Scala/Kotlin languages, and the choices the respective authors made along the way.
-
To get a better feeling of Kotlin I might pick it up for a small project, at which point I will surely blog about it some more..
from the list select the Android version your afer, right now its Android 4.0.3 (API 15) (I also installed Android 2.3.3), click on Install packages to get the installation started
-
-
setup your env echo "export ANDROID_HOME=~/Applications/android-sdk-linux/" >> ~/.bashrc
-
-
refresh your currect session source ~/.bashrc
-
-
-
Adding SBT support to IntelliJ
-
open File > Settings > Plugin
-
-
click on Browser repositories
-
-
find SBT, right click and select Download and install
create folder ~/bin/ (if you dont have ~/bin in your PATH, then you can do this: echo "PATH=$PATH:~/bin" >> ~/.bashrc (refresh your currect session source ~/.bashrc)
Finally, now that you have all the tools you need, proceed to creating a new project
-
run the android plugin project setup tool: ~/bin/g8 jberkel/android-app, follow the onscreen questions (or press return for defaults) this will create the project folder and create the files needed to build an android project
If you need to test/mock 3rd party API’s / a crawler / external web service, and you dont want to use some fancy DI framework (I hate the magic in DI frameworks, I use constructor DI instead..), anyway, here’s a helper class I put togather to that end
-
If you haven’t got a chance to try out Spray, you totally should, its a high performance web server built on top of Netty and Akka
-
Spray has a very nice HTTP API. Few things I liked:
-
-
HTTP method mapping spray.http.HttpMethods - GET, POST etc..
-
HttpRequest and HttpResopnse are basically wrappers around data you would expect to have in a request and response data
-
spray.http.Uri wrapper class around what you would expect to find in the URI
-
it really is just an Akka Actor, with just the right amount of control exposed and the rest well hidden behind its nice APIs
import akka.actor._ import akka.io.IO import java.util.concurrent.atomic.AtomicInteger import spray.can.Http import spray.http.{Uri, HttpMethod, HttpResponse, HttpRequest} objectHelpers{ lazyimplicitval testSystem = ActorSystem("helpers-test-system") // used to track the last assigned port val lastPort = newAtomicInteger(20000) // a single expected triplet of method+uri+body typeReplay= (HttpMethod,Uri.Path,String) // factory method to get a replater to play around with defwebReplayer(replay : List[Replay]) : (ActorRef,Int) = { // everytime the factory is used a new port is assigned, // so many instances can co-exist and tests can run in parallel without getting // on eachother's toes val newPort = lastPort.addAndGet(1) // instantiate a new Spray server. Yes, its that simple! val newServer = testSystem.actorOf(Props(newHelpers.ReplayerActor(replay))) IO(Http) ! Http.Bind(newServer, interface = "127.0.0.1", port = newPort) // return a pair of the server instance and the assigned port // the newPort should be used by client code to construct the remote URI with (newServer,newPort) } // object to use to tell the server to shutdown.. objectShutTestServer // a concrete implementation for a Spray Actor to listen to incoming HttpRequest(..) messages // provided a list of messages to be matched against arrived and content to be sent back classReplayerActor(var replay :List[Replay]) extendsActor{ var ioServer : ActorRef = null overridedefreceive: Receive = { case _ : Http.Connected => sender ! Http.Register(self) ioServer = sender() // this is basically a "catch all" HttpRequest pattern caseHttpRequest(method, uri,_,_,_) => // a test to see that this is indeed expected to happen at this point if (!replay.isEmpty && replay.head._1 == method && uri.path == replay.head._2){ sender ! HttpResponse(entity = replay.head._3) replay = replay.tail } else { sender ! HttpResponse(status = 400, entity = "was expecting a different request at this point") } caseShutTestServer => ioServer ! PoisonPill case _ => //ignore } } }
-
And now a cocrete client code example to use this tool
// some random content I would like to receive in this scenario (or test..) val content = s""" |<html> |<head> |<title>sometitle</title> |</head> |<body>some body</body> |</html> """.stripMargin // we use the factory method to get an instance of "replayer" to play around with val (_testWebServer,port) = Helpers.webReplayer( List( (GET,Uri.Path("/somesite"),content) ) ) // and where ever in your actual code, use the port we got back above and have it "hit" the server val whatEver = YourHTTPGetter.fetch(s"http://127.0.0.1:$port/somesite") assert(whatEver == content)
If you need to test/mock 3rd party API’s / a crawler / external web service, and you dont want to use some fancy DI framework (I hate the magic in DI frameworks, I use constructor DI instead..), anyway, here’s a helper class I put togather to that end
-
If you haven’t got a chance to try out Spray, you totally should, its a high performance web server built on top of Netty and Akka
-
Spray has a very nice HTTP API. Few things I liked:
-
-
HTTP method mapping spray.http.HttpMethods - GET, POST etc..
-
HttpRequest and HttpResopnse are basically wrappers around data you would expect to have in a request and response data
-
spray.http.Uri wrapper class around what you would expect to find in the URI
-
it really is just an Akka Actor, with just the right amount of control exposed and the rest well hidden behind its nice APIs
import akka.actor._ import akka.io.IO import java.util.concurrent.atomic.AtomicInteger import spray.can.Http import spray.http.{Uri, HttpMethod, HttpResponse, HttpRequest} objectHelpers{ lazyimplicitval testSystem = ActorSystem("helpers-test-system") // used to track the last assigned port val lastPort = newAtomicInteger(20000) // a single expected triplet of method+uri+body typeReplay= (HttpMethod,Uri.Path,String) // factory method to get a replater to play around with defwebReplayer(replay : List[Replay]) : (ActorRef,Int) = { // everytime the factory is used a new port is assigned, // so many instances can co-exist and tests can run in parallel without getting // on eachother's toes val newPort = lastPort.addAndGet(1) // instantiate a new Spray server. Yes, its that simple! val newServer = testSystem.actorOf(Props(newHelpers.ReplayerActor(replay))) IO(Http) ! Http.Bind(newServer, interface = "127.0.0.1", port = newPort) // return a pair of the server instance and the assigned port // the newPort should be used by client code to construct the remote URI with (newServer,newPort) } // object to use to tell the server to shutdown.. objectShutTestServer // a concrete implementation for a Spray Actor to listen to incoming HttpRequest(..) messages // provided a list of messages to be matched against arrived and content to be sent back classReplayerActor(var replay :List[Replay]) extendsActor{ var ioServer : ActorRef = null overridedefreceive: Receive = { case _ : Http.Connected => sender ! Http.Register(self) ioServer = sender() // this is basically a "catch all" HttpRequest pattern caseHttpRequest(method, uri,_,_,_) => // a test to see that this is indeed expected to happen at this point if (!replay.isEmpty && replay.head._1 == method && uri.path == replay.head._2){ sender ! HttpResponse(entity = replay.head._3) replay = replay.tail } else { sender ! HttpResponse(status = 400, entity = "was expecting a different request at this point") } caseShutTestServer => ioServer ! PoisonPill case _ => //ignore } } }
-
And now a cocrete client code example to use this tool
// some random content I would like to receive in this scenario (or test..) val content = s""" |<html> |<head> |<title>sometitle</title> |</head> |<body>some body</body> |</html> """.stripMargin // we use the factory method to get an instance of "replayer" to play around with val (_testWebServer,port) = Helpers.webReplayer( List( (GET,Uri.Path("/somesite"),content) ) ) // and where ever in your actual code, use the port we got back above and have it "hit" the server val whatEver = YourHTTPGetter.fetch(s"http://127.0.0.1:$port/somesite") assert(whatEver == content)
-
Profit!
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/themes/landscape/Gruntfile.js b/themes/landscape/Gruntfile.js
new file mode 100644
index 0000000..59fd5df
--- /dev/null
+++ b/themes/landscape/Gruntfile.js
@@ -0,0 +1,46 @@
+module.exports = function(grunt){
+ grunt.initConfig({
+ gitclone: {
+ fontawesome: {
+ options: {
+ repository: 'https://github.com/FortAwesome/Font-Awesome.git',
+ directory: 'tmp/fontawesome'
+ },
+ },
+ fancybox: {
+ options: {
+ repository: 'https://github.com/fancyapps/fancyBox.git',
+ directory: 'tmp/fancybox'
+ }
+ }
+ },
+ copy: {
+ fontawesome: {
+ expand: true,
+ cwd: 'tmp/fontawesome/fonts/',
+ src: ['**'],
+ dest: 'source/css/fonts/'
+ },
+ fancybox: {
+ expand: true,
+ cwd: 'tmp/fancybox/source/',
+ src: ['**'],
+ dest: 'source/fancybox/'
+ }
+ },
+ _clean: {
+ tmp: ['tmp'],
+ fontawesome: ['source/css/fonts'],
+ fancybox: ['source/fancybox']
+ }
+ });
+
+ require('load-grunt-tasks')(grunt);
+
+ grunt.renameTask('clean', '_clean');
+
+ grunt.registerTask('fontawesome', ['gitclone:fontawesome', 'copy:fontawesome', '_clean:tmp']);
+ grunt.registerTask('fancybox', ['gitclone:fancybox', 'copy:fancybox', '_clean:tmp']);
+ grunt.registerTask('default', ['gitclone', 'copy', '_clean:tmp']);
+ grunt.registerTask('clean', ['_clean']);
+};
\ No newline at end of file
diff --git a/themes/landscape/LICENSE b/themes/landscape/LICENSE
new file mode 100644
index 0000000..9ce4d32
--- /dev/null
+++ b/themes/landscape/LICENSE
@@ -0,0 +1,7 @@
+Copyright (c) 2013 Tommy Chen
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
\ No newline at end of file
diff --git a/themes/landscape/README.md b/themes/landscape/README.md
new file mode 100644
index 0000000..6567104
--- /dev/null
+++ b/themes/landscape/README.md
@@ -0,0 +1,111 @@
+# Landscape
+
+A brand new default theme for [Hexo].
+
+- [Preview](http://hexo.io/hexo-theme-landscape/)
+
+## Installation
+
+### Install
+
+``` bash
+$ git clone https://github.com/tommy351/hexo-theme-landscape.git themes/landscape
+```
+
+**Landscape requires Hexo 2.4 and above.**
+
+### Enable
+
+Modify `theme` setting in `_config.yml` to `landscape`.
+
+### Update
+
+``` bash
+cd themes/landscape
+git pull
+```
+
+## Configuration
+
+``` yml
+# Header
+menu:
+ Home: /
+ Archives: /archives
+rss: /atom.xml
+
+# Content
+excerpt_link: Read More
+fancybox: true
+
+# Sidebar
+sidebar: right
+widgets:
+- category
+- tag
+- tagcloud
+- archives
+- recent_posts
+
+# Miscellaneous
+google_analytics:
+favicon: /favicon.png
+twitter:
+google_plus:
+```
+
+- **menu** - Navigation menu
+- **rss** - RSS link
+- **excerpt_link** - "Read More" link at the bottom of excerpted articles. `false` to hide the link.
+- **fancybox** - Enable [Fancybox]
+- **sidebar** - Sidebar style. You can choose `left`, `right`, `bottom` or `false`.
+- **widgets** - Widgets displaying in sidebar
+- **google_analytics** - Google Analytics ID
+- **favicon** - Favicon path
+- **twitter** - Twiiter ID
+- **google_plus** - Google+ ID
+
+## Features
+
+### Fancybox
+
+Landscape uses [Fancybox] to showcase your photos. You can use Markdown syntax or fancybox tag plugin to add your photos.
+
+```
+
+
+{% fancybox img_url [img_thumbnail] [img_caption] %}
+```
+
+### Sidebar
+
+You can put your sidebar in left side, right side or bottom of your site by editing `sidebar` setting.
+
+Landscape provides 5 built-in widgets:
+
+- category
+- tag
+- tagcloud
+- archives
+- recent_posts
+
+All of them are enabled by default. You can edit them in `widget` setting.
+
+## Development
+
+### Requirements
+
+- [Grunt] 0.4+
+- Hexo 2.4+
+
+### Grunt tasks
+
+- **default** - Download [Fancybox] and [Font Awesome].
+- **fontawesome** - Only download [Font Awesome].
+- **fancybox** - Only download [Fancybox].
+- **clean** - Clean temporarily files and downloaded files.
+
+[Hexo]: http://zespia.tw/hexo/
+[Fancybox]: http://fancyapps.com/fancybox/
+[Font Awesome]: http://fontawesome.io/
+[Grunt]: http://gruntjs.com/
diff --git a/themes/landscape/_config.yml b/themes/landscape/_config.yml
new file mode 100644
index 0000000..3a8edb1
--- /dev/null
+++ b/themes/landscape/_config.yml
@@ -0,0 +1,26 @@
+# Header
+menu:
+ Home: /
+ Archives: /archives
+rss: /atom.xml
+
+# Content
+excerpt_link: Read More
+fancybox: true
+
+# Sidebar
+sidebar: right
+widgets:
+- category
+- tag
+- tagcloud
+- archive
+- recent_posts
+
+# Miscellaneous
+google_analytics: 'UA-28741273-1'
+favicon: /favicon.png
+twitter:
+google_plus:
+fb_admins:
+fb_app_id:
diff --git a/themes/landscape/layout/_partial/after-footer.ejs b/themes/landscape/layout/_partial/after-footer.ejs
new file mode 100644
index 0000000..fdb2970
--- /dev/null
+++ b/themes/landscape/layout/_partial/after-footer.ejs
@@ -0,0 +1,24 @@
+<% if (config.disqus_shortname){ %>
+
+<% } %>
+
+
+
+<% if (theme.fancybox){ %>
+
+
+<% } %>
+
+<%- js('js/script') %>
\ No newline at end of file
diff --git a/themes/landscape/layout/_partial/archive-post.ejs b/themes/landscape/layout/_partial/archive-post.ejs
new file mode 100644
index 0000000..36f2cc3
--- /dev/null
+++ b/themes/landscape/layout/_partial/archive-post.ejs
@@ -0,0 +1,8 @@
+
+
+
\ No newline at end of file
diff --git a/themes/landscape/layout/_partial/mobile-nav.ejs b/themes/landscape/layout/_partial/mobile-nav.ejs
new file mode 100644
index 0000000..3e1c5bd
--- /dev/null
+++ b/themes/landscape/layout/_partial/mobile-nav.ejs
@@ -0,0 +1,5 @@
+
\ No newline at end of file
diff --git a/themes/landscape/layout/_partial/post/category.ejs b/themes/landscape/layout/_partial/post/category.ejs
new file mode 100644
index 0000000..db2ed48
--- /dev/null
+++ b/themes/landscape/layout/_partial/post/category.ejs
@@ -0,0 +1,10 @@
+<% if (post.categories && post.categories.length){ %>
+
+<% } %>
\ No newline at end of file
diff --git a/themes/landscape/layout/_partial/post/date.ejs b/themes/landscape/layout/_partial/post/date.ejs
new file mode 100644
index 0000000..99470c8
--- /dev/null
+++ b/themes/landscape/layout/_partial/post/date.ejs
@@ -0,0 +1,3 @@
+
+
+
\ No newline at end of file
diff --git a/themes/landscape/layout/_partial/post/gallery.ejs b/themes/landscape/layout/_partial/post/gallery.ejs
new file mode 100644
index 0000000..ac264cc
--- /dev/null
+++ b/themes/landscape/layout/_partial/post/gallery.ejs
@@ -0,0 +1,11 @@
+<% if (post.photos && post.photos.length){ %>
+
+ <% } %>
+<% } %>
\ No newline at end of file
diff --git a/themes/landscape/layout/_partial/sidebar.ejs b/themes/landscape/layout/_partial/sidebar.ejs
new file mode 100644
index 0000000..c1e48e5
--- /dev/null
+++ b/themes/landscape/layout/_partial/sidebar.ejs
@@ -0,0 +1,5 @@
+
\ No newline at end of file
diff --git a/themes/landscape/layout/_widget/archive.ejs b/themes/landscape/layout/_widget/archive.ejs
new file mode 100644
index 0000000..9a428ba
--- /dev/null
+++ b/themes/landscape/layout/_widget/archive.ejs
@@ -0,0 +1,8 @@
+<% if (site.posts.length){ %>
+
+
Archives
+
+ <%- list_archives() %>
+
+
+<% } %>
\ No newline at end of file
diff --git a/themes/landscape/layout/_widget/category.ejs b/themes/landscape/layout/_widget/category.ejs
new file mode 100644
index 0000000..7836a27
--- /dev/null
+++ b/themes/landscape/layout/_widget/category.ejs
@@ -0,0 +1,8 @@
+<% if (site.categories.length){ %>
+
+
Categories
+
+ <%- list_categories() %>
+
+
+<% } %>
\ No newline at end of file
diff --git a/themes/landscape/layout/_widget/recent_posts.ejs b/themes/landscape/layout/_widget/recent_posts.ejs
new file mode 100644
index 0000000..3986ffb
--- /dev/null
+++ b/themes/landscape/layout/_widget/recent_posts.ejs
@@ -0,0 +1,14 @@
+<% if (site.posts.length){ %>
+
+<% } %>
\ No newline at end of file
diff --git a/themes/landscape/layout/_widget/tag.ejs b/themes/landscape/layout/_widget/tag.ejs
new file mode 100644
index 0000000..c163c41
--- /dev/null
+++ b/themes/landscape/layout/_widget/tag.ejs
@@ -0,0 +1,8 @@
+<% if (site.tags.length){ %>
+
+
Tags
+
+ <%- list_tags() %>
+
+
+<% } %>
\ No newline at end of file
diff --git a/themes/landscape/layout/_widget/tagcloud.ejs b/themes/landscape/layout/_widget/tagcloud.ejs
new file mode 100644
index 0000000..4b82283
--- /dev/null
+++ b/themes/landscape/layout/_widget/tagcloud.ejs
@@ -0,0 +1,8 @@
+<% if (site.tags.length){ %>
+
+
Tag Cloud
+
+ <%- tagcloud() %>
+
+
+<% } %>
\ No newline at end of file
diff --git a/themes/landscape/layout/archive.ejs b/themes/landscape/layout/archive.ejs
new file mode 100644
index 0000000..52f9b21
--- /dev/null
+++ b/themes/landscape/layout/archive.ejs
@@ -0,0 +1 @@
+<%- partial('_partial/archive', {pagination: config.archive, index: true}) %>
\ No newline at end of file
diff --git a/themes/landscape/layout/category.ejs b/themes/landscape/layout/category.ejs
new file mode 100644
index 0000000..3ffe252
--- /dev/null
+++ b/themes/landscape/layout/category.ejs
@@ -0,0 +1 @@
+<%- partial('_partial/archive', {pagination: config.category, index: true}) %>
\ No newline at end of file
diff --git a/themes/landscape/layout/index.ejs b/themes/landscape/layout/index.ejs
new file mode 100644
index 0000000..60a2c68
--- /dev/null
+++ b/themes/landscape/layout/index.ejs
@@ -0,0 +1 @@
+<%- partial('_partial/archive', {pagination: 2, index: true}) %>
\ No newline at end of file
diff --git a/themes/landscape/layout/layout.ejs b/themes/landscape/layout/layout.ejs
new file mode 100644
index 0000000..1524a62
--- /dev/null
+++ b/themes/landscape/layout/layout.ejs
@@ -0,0 +1,18 @@
+<%- partial('_partial/head') %>
+
+