Saturday, November 25, 2017

Udacity Lab: Simulated Annealing Lab Results

Introduction

I'm currently doing a course at Udacity and I'm using this post to share some results with my peers. We were asked to analyse how changing certain properties of the Simulated Annealing Algorithm would change the solutions of the Traveling Salesman Problem. I produced some histograms which I found interesting enough to share. My apologies if I don't explain anything further.

Changing the Number of Cities

Nobs 100, Min 1426, Max 2450, Mean 1836, Var 63180, Skewness 0.622


Nobs 100, Min 4487, Max 7316, Mean 5840, Var 277449, Skewness 0.0098

By increasing the number of cities in the second graph we increase the complexity of the problem and we see that the algorithm most of the time only finds a sub optimal solution. The optimal solution is at the Min-Value to the left and only with 10 cities we see a graph that is skewed to the right. 

Changing Temperatur

Nobs 100, Min 1426, Max 2501, Mean 1895, Var 86949, Skewness 0.439

Nobs 100, Min 1426, Max 2439, Mean 1853, Var 76739, Skewness 0.391

Nobs 100, Min 1426, Max 2427, Mean 1859, Var 74511, Skewness 0.363

Nobs 100, Min 1426, Max 2436, Mean 1849, Var 73730, Skewness 0.271

Nobs 100, Min 1426, Max 2564, Mean 2040, Var 92452, Skewness -0.450

We see that choosing a very high temperature has not a huge effect. The first two histograms look the same even though the temperature on the first one was significantly higher. But we see in the last two histograms that choosing a too low temperature is not good as we start to skew to the left.

Changing Alpha

Nobs 100, Min 1426, Max 2368, Mean 1693, Var 37448, Skewness 0.587

Nobs 100, Min 1434, Max 2465, Mean 1870, Var 63441, Skewness 0.335

Nobs 100, Min 1426, Max 2548, Mean 2007, Var 82375, Skewness -0.379


Nobs 100, Min 1426, Max 2608, Mean 2065, Var 75604, Skewness -0.545

Nobs 100, Min 1676, Max 2729, Mean 2198, Var 60915, Skewness -0.283

We clearly see that choosing a high alpha value is best as the graph skews to the right while the variance is still the smallest. A high alpha significantly increase the run time therefore alpha and run time have to be carefully balanced.




Friday, April 1, 2016

Food Hero on wit.ai

The guys at wit.ai list Food Hero on their website now. That's pretty cool.

Have a look here and scroll down to Food Hero.

Friday, October 30, 2015

Food Hero is on the Apple App Store

I've been feverishly working on my first App and finally it's there... on the world famous Apple App Store. It arrived in heaven so to speak.

Food Hero is a restaurant guide with which you interact through a conversation. I always found a conversation based approach very interesting since it opens up a lot of new options which traditional guides, for example yelp, trip advisor, google etc..., don't have. It's an experiment and I have no idea where it goes next. Is it just a curiosity? Is searching for restaurants the right thing to do? Will Siri, Cortana, Google Now & co. do what I envision with Food Hero in future?

Food Hero has been trained with what I imagined people would do with it. I know reality will be much different. And that's my next aim. Learning what people actually do with it.

Download it, try it out and please let Food Hero collect usage data since that is really what I'm after at the moment. If you are concerned about your privacy please check out Food Hero's privacy policy.

For more information please visit:

Tuesday, June 25, 2013

Why neography-batch is useful to me

I'm working on a ruby on rails project and I decided to use neo4j as one of the data stores. The software is about writing medial case reports and all reports are internally deconstructed into their syntactical and semantic structure. During that process, words are stored as neo4j-nodes (organised in sentences, chapters and reports). Words are also connected to other words trough neo4j-relations (e.g.: preposition, adjective-modifier, possession-modifier etc..) and sentences are connected to medical concepts (e.g.: Retroperitoneal hemorrhage) and concepts can be connected to other concepts.

This process goes to show that, when saving a report, a considerable amount of nodes and relationships have to be created, updated or delete. I tried following approaches to access the neo4j-server:

  1. Neo4j Java API via Rjb (Ruby Java Bridge): This approach was too slow since the Java API is fine-grained and each call into the Java API had to be bridged between ruby and java. Because I planned to deploy the software in the cloud (Heroku and the neo4j-plugin) this approach wouldn't have worked anyway.
  2. Neograpyh: This was a better solution since Neography is a ruby-gem which accesses neo4j trough its REST API. In order to have a good performance I exclusively used Cypher queries and Batches. Everything else was too fine-grained for my purposes. Additionally, I used batches to implement transactions. I aggregated all 'commands' into one batch which was sent over the network as a whole and had it executed by the neo4j-server in one transaction. This solution enabled me to define reports as an Aggregate and to always modify them in one batch. 
However, in the course of creating and/or modifying a report it proved difficult to use Neography's native batch functionality. The results produced by different parts of my software couldn't be easily aggregated into one single batch. Therefore, I wrote a tiny extension to Neography called neography-batch. The neography-batch helps composing several batches into larger batches and referencing specific 'commands' in a batch from the same batch or from another batch. If you are interested, please see neography-batch on github or RubyGems.org.

Friday, October 21, 2011

Short abstract of HTML 5

I've recently finished reading Pro HTML5 Programming. The book includes good examples, it is easy to read, and it's a nice introduction to HTML5. Some examples unfortunately didn't work but I guess they were based on an older version of the HTML5 specifications which wasn't supported by my browsers any more.

Following a brief summary of the new HTML5 features ordered by my view of their importance:
  • HTML5 WebSocket: This is a lightweight duplex communication channel between server and websites and it offers a faster way to send small messages back and forth. A WebSocket can just be opened once the http communication has already been established ("Upgrading the http-connection to WebSocket")
  • HTML5 Web Workers: A WebWorker is setup by a JavaScript and executes another JavaScript in parallel. Communication to and from WebWorkers is preferably implemented using 'Cross Document Messaging' or messaging via the Web Storage.
  • HTML5 Web Storage: Stores key/value-pairs in either a session store or a local store. The stores are isolated by their origin (e.g: www.evil.com can't access values from www.example.com). There are events published by the store which are fired when a value changes. Those events can also be used to implement messaging between websites.
  • HTML5 Canvas: A simple API to render 2D-drawings from within the browser using Java-Script. It's also possible to read and modify parts of the canvas using bitmaps.
  • Communication APIs: 'Cross Document Messaging' allows sending messages and events between parts of a website even they are not from the same origin. (which hasn't been possible due to security concerns before). 'XMLHttpRequest Level 2' allows the same for communication to the server. It means a website originating from 'example.net' can send XMLHttpRequests to different origins like 'example.net' and 'example.com' at the same time. The import part is that both sender and receiver of 'cross messages' have to be configured accordingly otherwise the communication is not possible due to security constraints.
  • HTML5 Geolocation: A simple API to access the browser's geographical location. The method of how the location is determined is hidden away by the API. Depending on the kind of browser and hardware the accuracy of the measurement can vary greatly.
  • HTML5 Audio and Video: Embeds audio and video natively in HTML. Defines controls to play audio and video. Audio and video editing is not included.
  • HTML5 Forms API: A bunch of new html tags and attributes that provide us with better semantics and therefore enable the browser to render more advanced or different controls. (e.g a control that just accepts E-mail addresses)
I'm really looking forward seeing where HTML5 will take us and what frameworks and paradigms will evolve around it. Will websites become more like traditional rich clients now?



Saturday, July 30, 2011

Typing effort analyzer in Ruby

I've been working on RePhraser over the course of the last 6 months. RePhraser is a piece of software which aims to help professionals in writing repetitive texts quickly. Unfortunately I haven't been able to make it available to a broader audience yet mainly due to the fact that it only works reliably in Internet Explorer and that there are still some basic features missing. Even so, I've uploaded a short demo here.


RePhraser assists you by anticipating and displaying words while you are typing them. Imagine you being a physician who writes the word "immunocytochemically". You would start to type "immu" whereas RePhraser would bring up all the words that start with "immu". There would be simple words like "immune", "immunogen" or more complicated words like the aforementioned "immunocytochemically". In fact there are a lot of words that starts with "immu". RePhraser would be pretty useless if it displayed easy words like "immune" or "immunogen" at this point because professional writers type those words much faster than RePhraser could display them. To solve this problem RePhraser uses a typing effort model. The basic idea is to display words which are harder to type first and to ignore words which are easy to write. To do so RePhraser rates every word based on the carpalx typing effort model. The carpalx typing effort model takes account of things like weak fingers (like pinky and ring finger), travel distance of fingers, same-finger typing (e.g "uhm"), balanced hand-use vs. right-hand priority etc...


I implemented the carpalx typing effort model in ruby and published source code and Gem on https://github.com/Enceradeira/teanalyzer. The project is named Teanalyzer, which is an abbreviation of typing effort analyzer.


What do you think about Teanalyzer? Or are you interested in RePhraser? Please drop a line or contact me!

Saturday, March 27, 2010

My days at QCon 2010, London

This was my first visit at a QCon-conference and I was quite excited to see some of the 'famous' speakers in the world of software development. There were such interesting talks on the schedule that I even missed out on the talks of Eric Evans and Martin Fowler. May be next year!
Generally I jumped between the different tracks but attended more than one talk from the tracks Software Craftsmanship, Functional programming and Irresponsible Architectures and Unusual Architects.

Following talks were my personal highlights (based on the speaker, importance to me, and what I learned from it):

Bad Code, Craftsmanship, Engineering, and Certification by Robert C.Martin. Nothing new, but nethertheless a very entertaining talk. The bad code video was phenomenal (especially with the depressing background music). He questioned if bad code is written because of deadlines, laziness,  boredom or even job security ('I´m the only guy that can maintain that!'). He also suggested to follow The Boy Scout Rule: "Always leave the campground cleaner than you found it". "The only way to go fast is to go well" was another of his wisdoms. And then he 'evangelized' agile practices like TDD, pair programming, CI etc... that leads us to what he calls 'Pride of Workmanship'.


Sharpening the Tools by Dan North. Good speaker and motivator. He reflected on the way how we learn things (the Dreyfuss model) and that therefore we are in everything that we learn a 'novice', 'advanced beginner', 'competent', 'proficient' or an 'expert'. He suggested that we have always to renew our skills due to the continuous development of new and more effective techniques in the area of software development:
  1. Practise the basics
  2. Learn from other people
  3. Understand trends
  4. Share knowledge
  5. Maintain your toolbox ('Some tools are timeless, some are disposable')
  6. Learn how to learn
Not Only SQL: Alternative Data Persistence and Neo4J by Emil Eifrém. It was the first time that I attended a talk about this subject and I was very curious about it. I realised soon that it's all about 'scalability'. Like: ´what kind of datastore to you need if you are building a twitter-like-application´. Relational databases are strong with well structured not very complex data (e.g salary-list). NoSQL-datastores are better with more dynamically defined, complex data (e.g: persons and their different relationships to each other). I learned that this is not the end of relational databases but that there are now other techniques available when it comes to storing large amount of complex,dynamic data. I know now that a query like 'all friends of Peter that also know Sarah' can be much more efficient and easier be build with a NoSQL-datastore (especially graph-DB's).

The Joy of Testing by John Hughes. A very refreshing but a little bit academic talk about TDD. The key idea is to abstract test-cases to a set of test-'properties'. Think about a number of test-cases that test a method. Wouldn't it be nice to reduce those test-cases to one unique description that could be run by a test-runner and that this test-runner would find edge-cases that you have never thought about it? I'm not (yet) able to apply this to my daily work but I'm still thinking about it.

Kanban - Crossing the line, pushing the limit or rediscovering the agile vision? by Jesper Boeg. A good talk given by a convinced Kanban-evangelist. It was very interesting how deep they integrate product-owner, buisness-analysts and tester into the development process ('developers helps out business-analyst to write down stories if there is a temporary bottleneck'). Sounds like 'extreme scrumming' to me! (very short full development-cycles). I really like the idea of helping out each other so that we can cope with overloaded/under-staffed testers, productowner and business-analysts.

Command-Query Responsibility Segregation by Udi Dahan. Generally he was saying that displayed data (query-part) could completely be decoupled from the persisted data and on the other hand updating the persisted data (command-part) could be done using a much more sophisticated model (e.g: a strong domain-model or a asynchronous event queue etc...). I already knew about this style but new to me was:
  • It's about how we build the UI and what business-services we offer to a user. I got the impression we should display less raw-data (e.g: less data-grids) and let the software behave more intelligent.
  • We could deliberately display stale data (e.g: display a account-list with title 'account-balance as it was on 25.4.2010 at 14:34h'). He says that this is usually no problem to a user.
  • We could try to build much more valuable commands (e.g: a reservation-system that can ´book the best seats for a group of 12 persons were no person must sit alone' instead of letting the user to choose the seats by itself) 
Scenario-Driven Development by Ben Butler-Cole (Unfortunately no slides available). In his talk he was suggesting not to do integration/acceptance-testing on a 'per-story' basis. He mentioned that this could lead to brittle tests and to maintenance problems. He suggested to identify a small number of 'key-scenarios' that spam in their final version several stories. A key-scenario for a banking-application could be: 'Poor guy wants to pay a bill´ that includes the stories 'User pays bill' (that is rejected due to insufficient balance),  'User applies for a overdraft', 'Clerk manages overdraft application', 'User pays bill' (that is not anymore rejected). Scenarios are developed incrementally and extended when a story is going to be implemented. He also showed Twist that was developed by Thoughtworks to support Scenario-Driven Development. An interesting thing is that Twist puts several layer on the top of the application layer, helping to manage reusability and maintainability of the tested UI. These layers are called Scenario-Layer (the scenarios written with Twist, Workflow-Layer (commands like ´go to homepage´) and the Application-model (abstracting the underlying technology with drivers like Selenium). The Workflow-Layer and the Application-model have to be written in Java and enjoy therefore all advantages of a modern programming-language (refactoring,abstraction, object-orientation etc...)

I also attended following less interesting talks:
  • Functional Approaches To Parallelism and Concurrency
  • Demystifying monads
  • Living and working with aging software
  • The Counterintuitive Web
  • Patterns for the People
  • Transactions: Over Used or Just Misunderstood?
  • Fighting Layout Bugs
  • Test-Driven Development of Asynchronous Systems
  • Data Presentation in a Web App: The Journey of a Startup
  • Death by accidental complexity

Thursday, October 22, 2009

Sample Application with White,WPF,TDD and DDD

I'm currently thinking about what we could do better on a next project avoiding some technical issues that we have had in our latest project. Some of the issues were

  1. The Passive View implementation for our Win-Form client was too costly. The design of it wasn't intuitive and straight forward at all.
  2. The Passive View implementation didn't allow us to unit-test the UI. Even we were able to test the presenters we still couldn't test the event- and data binding automatically .
  3. Our initial plan to do acceptance- and integrationtesting with TestComplete failed. No tests have ever been written with TestComplete.
  4. Our testing-suite runs too slow. The whole suite was too slow and even running one single test took too much time.
  5. We struggled with Aggregates and domain logic leaked sometimes into the presentation layers.
  6. The application is too slow. This was caused due the fact that we couldn't react appropriately when we were forced to move our 2-tier implementation to a 3-tier implementation.
I've recently written a sample application to find answers to some of those problems. You can download the application from this site (Click 'Save file to your PC' at the end of the site).


I'm going to explain my thoughts in detail maybe later, but the most interesting points in the sample application are:
  • WPF+White: White allows testing the UI right from inside the unit-test runner. Following some basic principals it allows to script the UI right from the c#-code. But I used White just to Unit-Test the UI and to write some basic integration tests. I don't believe that one should use White to do more than that otherwise he might get some serious maintenance issues. For details see OrderExploration.Test\UnitTest\Ui and OrderExploration.Test\Integration
  • Acceptance-Testing: I think acceptance-testing frameworks are a weak point of the .net-world. Either they require a lot of infrastructure (like wiki's) or they can't be used by "none technical's". NBehave is useless because it can't be used by "none technical's" and it doesn't produce readable code at all (That statement is maybe wrong with Version 0.4, but I didn't check). FitNess is far too complicated for a small project like the sample application. I tried StoryTeller as well, but there is not a lot of documentation available yet, so I gave up. So why not using pure old nunit-tests if there are no "none technical's"? For details see OrderExploration.Test\Acceptance
  • Persistence-Ignorance: There is just one test that hits the database (see OrderExploration.Test\Integration). All other tests (UI, unit und acceptance) run against stubs.
  • Test-Setups: To setup Unit- Acceptance- and Integrationtest differently I used SetupFixtures (I didn't know them before!). Lookout for classes with name "SetupFixture"
  • CQS (Command-Query Seperation) inspired by a Ian Cooper: I tried to separate querying data (mainly for displaying on view) and commanding (mainly for updating on aggregates). I'm not sure if I really got it right! For an example see method DataService.GetOrders() that queries data right from the database and see OrderService.Submit() that writes changes through the aggregate to the database.
  • Aggregates: See the class Order that defines an Aggregate-Root. I tried to build an expressive aggregate by separating the aggregate from its persisted structure (see class OrderRow) and its displayed structure (see class OrderDetailData). If there is an interest I would like to discuss that decision later. For now see method Order.AddTag() and think about: 1.) If the class Order were directly bound to the UI, how would you handle the fact that the user can add/remove tags several time but just the last state should be persisted and logged in the history. 2.) If the class Order were persisted directly, how would you handle the fact that tags are persisted as strings? (see Order.Tags in the database-scheme)
  • and StructureMap and AutoMapper ... of course!
Summary
  • White is cool and helps to (Unit)-test UI
  • Persistence Ignorance can be implemented and helps to make Unit-/UI-/Acceptance tests much faster
  • CQS (Command-Query-Separation), ViewModels, DomainModel and PersistenceModel add another layer of indirection that can help to solve tricky problems quite easly. However, these indirections can be a huge overkill for simple applications. I have no doubts that it's worth to have ViewModels (including all kind of MVP,MVC) but I'm not sure when it's worth to separate DomainModel and PersistenceModel. In the sample application, DDD (e.g Repository-Pattern) helped me to answer that question per Aggregate. In the sample application the Aggregate Order has a separated DomainModel and PersistenceModel. For the other Aggregates DomainModel and PersistenceModel are the same. Any thoughts about that?







    Monday, August 3, 2009

    IObservable ... it's so beautiful

    I've just watched Brian Beckman and Erik Meijer - Inside the .NET Reactive Framework (Rx) and it's again a great work of Erik, Wes and their team.

    It's a petty that I couldn't understand all that formalism but i enjoyed it like art and i'm impressed about the beauty and completness of their solution. That's the first mind-blowing feature of .NET 4.0!

    Sunday, July 26, 2009

    Object Role Stereotypes

    I had a short discussions with my college Jürg last week about that classes in object oriented design (especially DDD or RDD) can be classified regarding their responsibility and behaviour. Neither could I remember the name for that concept nor any of those classifications. Jürg mentioned that this is like stereotypes in UML. I looked it up now and he wasn't wrong. They are called 'Object Role Stereotypes'. Not that Object Role Stereotypes are defined by the UML specification but one could use stereotypes in UML to classify classes by their Object Role Stereotype. Here are the stereotypes that I found:

    • Controller – Controls application execution. Makes decisions and delegates to other classes.
    • Service Provider – Provides a service to other classes. Performs a calculation, computation, or executes business rules.
    • Interfacer – Communicates actions to other layers or systems.
    • Information Holder – Holds facts. Domain classes.
    • Structurer – Maintains relationships between other classes

    Sunday, March 15, 2009

    How to get the console output working with NBehave 0.4 and the ReSharper Testrunner

    When working with the latest build of NBehave 0.4 you may notice that there is no output written to the ReSharper Testrunner Output anymore.

    Here is how I did workaround the problem. I derived the specs from the following class and every thing was fine again:
    public class SpecBaseWithConsoleOutput: SpecBase
    {
    private EventHandler<EventArgs<MessageEventData>> addedHandler;
    private EventHandler<EventArgs<Scenario>> scenarioCreatedHandler;
    private EventHandler<EventArgs<Story>> storyCreatedHandler;
    public override void MainSetup()
    {
    base.MainSetup();
    addedHandler = (o, a) => Console.WriteLine(a.EventData.Message);
    scenarioCreatedHandler = (o, a) => Console.WriteLine(a.EventData.Title);
    storyCreatedHandler = (o, a) => Console.WriteLine(a.EventData.Title);
    Story.MessageAdded += addedHandler;
    Story.ScenarioCreated += scenarioCreatedHandler;
    Story.StoryCreated += storyCreatedHandler;
    }
    public override void MainTeardown()
    {
    Story.MessageAdded -= addedHandler;
    Story.ScenarioCreated -= scenarioCreatedHandler;
    Story.StoryCreated -= storyCreatedHandler;
    base.MainTeardown();
    }
    }

    Saturday, January 24, 2009

    Implementing and testing mapping code with AutoMapper

    I was thinking some months ago that it would be great to have a property mapper that eliminates tedious mapping code . I'm very happy that Jimmy Bogard and his team just have published a framework called AutoMapper that resolves that issue. I haven't tested it yet but the idea looks promising.

    Why would I use AutoMapper?
    • In order to implement better encapsulation and layering with less amount of coding
    • In order to unit-test mapping code using a simple 'Mapper.AssertConfigurationIsValid()'

    Sunday, January 11, 2009

    TypeMockAutoMocker on CodePlex

    I' ve just uploaded the TypeMockAutoMocker to http://www.codeplex.com/typemockautomocker

    How to use it:
    1.) Download TypeMockAutoMocker.dll
    2.) Copy it to your project and set an assembly reference to it
    2.) Set 'Copy local = true' on your TypeMock.dll
    3.) Have fun (see Link)

    StructureMap 2.5.1 and TypeMock 5.1.2 ist the only tested configuration at the moment.

    Friday, January 2, 2009

    AutoMocking Container with TypeMock Isolator

    I have implemented an AutoMocking container for the Isolator inspired by an Introduction to the AutoMocking container in StructureMap, the fact that our test setups usually are pretty messy and Jeremy's claim that TypeMock users probably don't care about dependency injection. But why do I want to use DI (Dependency Injection) in conjunctions with TypeMock? I don't want to fight a 'religious war' based on that question, but here is what I think:

    • TypeMock without DI lacks an explicit boundary that defines what the class under tests is and which classes are mocks or stubs. This is about solid design and to isolate parts of the software in a structured manner. It’s true that you can mock everything with TypeMock but this is just about mocking and not about software design.

    • DI with another mocking framework than TypeMock is like .NET without System.Reflection. I don't like to use System.Reflection but sometimes it's just the right thing. It’s the same with TypeMock. If everything (own code, legacy-code, external libraries etc.) had a solid design you wouldn't need it but sometime it makes the life so much easier.
    Example without Dependency Injection
    There is the class under test called ValidationViewPresenter and it has following dependencies:



    A a test without DI could look like:
    [Test]
    public void CanValidateCreditCardNumber()
    {
    // Arrange
    var viewMock = Isolate.Fake.Instance<ValidationView>();
    var customerServiceMock = Isolate.Fake.Instance<CustomerService>();
    var creditCardServiceMock = Isolate.Fake.Instance<CreditcardService>();
    var customer56 = new Customer {CreditCardNr = "5500 0001 0001 0001"};

    // Mock the (hidden) dependencies
    Isolate.Swap.NextInstance<ValidationView>().With(viewMock);
    Isolate.Swap.NextInstance<CustomerService>().With(customerServiceMock);
    Isolate.Swap.NextInstance<CreditcardService>().With(creditCardServiceMock);
    // Mock the calls to the dependencies
    Isolate.WhenCalled(()=>viewMock.CustomerNr).WillReturn(56);
    Isolate.WhenCalled(() => customerServiceMock.GetCustomer(56)).WillReturn(customer56);
    Isolate.WhenCalled(()=> creditCardServiceMock.ValidateCreditCard("5500 0001 0001 0001")).WillReturn(true);

    // Act
      var presenter = new ValidationViewPresenter();

    presenter.ValidateCreditCardOfCustomer();
    // Assert
    Isolate.Verify.WasCalledWithExactArguments(() => viewMock.IsValid = true);
    }

    Example with Dependency Injection
    After introducing constructor injection it looks like:
    [Test]
    public void CanValidateCreditCardNumber()
    {
    // Arrange
    var viewMock = Isolate.Fake.Instance<ValidationView>();
    var customerServiceMock = Isolate.Fake.Instance<CustomerService>();
    var creditCardServiceMock = Isolate.Fake.Instance<CreditcardService>();
    var customer56 = new Customer {CreditCardNr = "5500 0001 0001 0001"};
    // Inject dependencies manually
      var presenter = new ValidationViewPresenter(viewMock, customerServiceMock, creditCardServiceMock);

    Isolate.WhenCalled(()=> viewMock.CustomerNr).WillReturn(56);
    Isolate.WhenCalled(() => customerServiceMock.GetCustomer(56)).WillReturn(customer56);
    Isolate.WhenCalled(() => creditCardServiceMock.ValidateCreditCard("5500 0001 0001 0001")).WillReturn(true);

    // Act
    presenter.ValidateCreditCardOfCustomer();

    // Assert
    Isolate.Verify.WasCalledWithExactArguments(() => viewMock.IsValid = true);
    }

    Example with Automocking Container
    Let's eliminate the explicit constructor injection by introducing the TypeMockAutoMocker now:

    [Test]
    public void CanValidateCreditCardNumber()
    {
    // Arrange
    var customer56 = new Customer {CreditCardNr = "5500 0001 0001 0001"};
      var presenterMocker = new TypeMockAutoMocker<ValidationViewPresenter>(MockMode.AAA);

    var viewMock = presenterMocker.Get<IValidationView>();
    var customerServiceMock = presenterMocker.Get<ICustomerService>();
    var creditCardServcieMock = presenterMocker.Get<ICreditCardService>();

    Isolate.WhenCalled(() => viewMock.CustomerNr).WillReturn(56);
    Isolate.WhenCalled(() => customerServiceMock.GetCustomer(56)).WillReturn(customer56);
    Isolate.WhenCalled(() => creditCardServcieMock.ValidateCreditCard("5500 0001 0001 0001")).WillReturn(true);

    // Act
    presenterMocker.ClassUnderTest.ValidateCreditCardOfCustomer();

    // Assert
    Isolate.Verify.WasCalledWithExactArguments(() => viewMock.IsValid = true);
    }
    As you can see the AutoMocking container couldn’t eliminate a lot of complexity. This is due to the fact that the test itself is complex and there are a number of non-default preconditions.

    A better example with AutoMocking Container
    Following code tests the same method but with different preconditions again. But this time we can rely on the default behavior that was setup by the AutoMocking container. This means that all involved and not explicitly mocked interfaces or classes do have a default behavior.

    [Test]
    public void CanHandleNotExistingCustomer()
    {
    // Arrange
      var presenterMocker = new TypeMockAutoMocker<ValidationViewPresenter>(MockMode.AAA);

    var viewMock = presenterMocker.Get<IValidationView>();
    Isolate.WhenCalled(() => viewMock.CustomerNr).WillReturn(56);

    // Act
    presenterMocker.ClassUnderTest.ValidateCreditCardOfCustomer();

    // Assert
    Isolate.Verify.WasCalledWithExactArguments(() => viewMock.IsValid = false);
    }
    Conclusion
    There is a big chance that you can get lost using TypeMock and StructureMap in conjunction. The problem is that there are too many ways how they can be combined. However I hope that the TypeMockAutoMocker could help in a way that it's defining a common pattern for how to setup the tests and how to inject the mocks. Remember always:
    • Depended classes are instantiated by the AutoMocker automatically.
    • Depended interfaces and abstract classes are instantiated as mocks. These mocks do have a default behaviour where every method can be called.
    • Depended concrete classes are instantiated by calling its greediest constructor. They are not mocked.
    • To change the default behaviour you can use Inject() to register your own mock to the AutoMocker. It's how you can setup a mock manually.

    Sound’s like ‘convention over configuration’. And that makes life definitely easier.

    
    

    Sunday, December 14, 2008

    A Look Back at Our Scrum implementation

    I've been working at my current project for one year now and we are 'nearly done' with our first major release. I know that saying 'nearly done' has a little bit of a bad smell in the agile community but we couldn't do everything perfectly. But generally speaking, we did it pretty well and I'm happy how we could eliminate and master the difficulties of the project with scrum.

    We had the following situation at the beginning of the project:
    • A list of requirements in an excel sheet. The list summarized all the thoughts, requirements and promises in an informal way.
    • There was a file containing drawings, print-outs from the former legacy system, or other artefacts for each requirement.
    • The project was embedded in a much larger project, which was driven by Hermes. Hermes is a heavy weight project management process, commonly used by governmental organisations in Switzerland.
    • Our team consisting out of 6 software developers was hired to implement the software. We proposed to implement scrum and the customer agreed on that.
    • A contract with a fixed price that was signed a long time before the developers got involved

    It was not easy to push scrum from the development into the larger project organisation. There was a good support for our ideas as long there was no organizational change required. Fortunately we could find a pretty good product owner, but we never have had a scrum master. The role of the scrum master was poorly fulfilled by me as a developer.

    What worked well?

    A lot of things worked great and I'm sure that the project would have failed without an agile approach. Here is what has worked well (see glossary for details on scrum terminology):

    • User-Stories: After we had transformed the informal requirement list into user stories it was relatively easy to estimate it without knowing the details of each story. Even they were written by us developers it was better than working with the initial list.
    • Productbacklog: It gave us a very important planning tool, helping us to focus on the near future and driving the development sprint by sprint.
    • Sprints: The sprints helped us a lot to get the team and the customer focused on the tasks that needs to be done. Without focusing on the sprints and forgetting a little bit about the whole, we could never have done it.
    • Sprintplanning: Without this meeting we wouldn't have any chance to know what to implement in detail. The customer was surprised at the beginning that he had do discuss every detail of its initial requirement list again in the sprint planning.
    • Sprintreview: It was often the only opportunity to present our achievements and we always have had valuable feedback from the customer and the future users. Customer and users could get used to our development velocity and that took a lot of pressure away.
    • Adoption: The product owner discovered during the first two sprints that he needs to be better prepared for the sprint meetings. He started to consolidate the different opinions in additional meetings prior to the sprint planning.
    • Team forming: Event just two out of six developers have worked together before we never had major problems.

    What were the problems?

    • Scrummaster / Productowner: The roles were never officially assigned to persons, because the customer was confused in relation to the existing roles and responsibilities (project manager, solution architect and so on). There was also no budget available for 'non-programming' tasks.
    • Importance of the Productbacklog: The Productbacklog was built by the developers and was never owned by the product owner! Once it was initially built up it was used by the management to plan the whole year front up! Development was driven by the Productbacklog, but not testing, acceptance and fulfilment of the contract.
    • Testing: We couldn’t get things DONE. As the customer didn't test we started to do more functional and system testing. But after one year we do not have any story officially accepted by the customer. But due to the sprint-reviews I'm pretty confident that the customers will accept it after some minor changes.
    • Sprint-Retrospectives: the meeting was always held by the team but not attended by the product owner, nor attended by another representative of the management. Therefore just team internal impediments could be addressed and resolved. No organizational impediments were resolved.
    • Meetings: Due to the fact that the whole process was not coached by a scrum master, the meetings usually were ineffective and lasted too long. I, as developer and interim scrum master, was too involved to act as a coach for both the team and the customer.
    • Responsibility: There were few developers in our team that couldn't follow the process at all. They were not able to catch up the information at the sprint planning nor did they take the responsibility to fulfil the tasks they assigned themselves. We lost a lot of our performance here as senior team members had to coach them. Those weaker developers could not catch up as every thing went far too fast for them.

    Conclusion
    I'm still saying that agile development is the best practice I know. In my career as a software developer I sat too many times in my office trying to imagine how I could fulfil some requirements that I didn't understand. Today we have a simple process that brings customer and team together and that's much better for everybody. I'm thinking agile development is a process for people that want to take responsibility. If you don't have people that take responsibility you might be better to follow more heavy weight processes were every thing is predetermined and fixed. But everything you can't foresee will cost you a lot of money! It will cost you much more than the salaries of a good scrum master and good developers.

    Thursday, October 16, 2008

    Reflecting about how we implement aggregates

    How we implement aggregates

    Since the beginning of my current project we tried to apply the ideas of structuring our business-logic into aggregates. We started to implement the aggregates using the following guidelines:

    • define an aggregate root and implement it as a public class
    • define the internal classes of an aggregate and implement it as a public class
    • implement a repository that can query and return aggregate roots of this class
    • try to avoid accessing the internals (like traversing from aggregate root to an internal class) from outside the aggregate
    • put as much as possible of the business-logic into the aggregate root

    Following this rules we had some sort of guidelines about where to locate the business-logic and when to implement a repository. I think it was worth to have the aggregates. It was a valuable concept when we had to change our application from a 2-tier to a 3-tier architecture.

    Nevertheless we always had following problems with aggregates:

    • Most of the time we had code in the presentation-layer that accessed the internals of an aggregate. It appeared often in case where there was an one-to-one data binding. For instance, in a master-detail view, where the master was the aggregate root an the detail an internal entity of the aggregate. (e.g. editing invoice and invoice position)
    • There were relations on the data model that do not comply the aggregate rules. In such a case we often had a nice definition of an aggregate, however there was sometimes a nasty dependency. It happend that an internal entity was depending on another entity in another aggregate. We agreed that this is not bad and that it can be abstracted to a dependency between the two involved aggregates.
    And what happened:

    • Since the internals could be accessed, the business-logic begun to spread out into the presentation layer. It's impossible always to have the discipline to comply the aggregate rules and all of us violated it from time to time.
    • We didn't centralize the creational-logic. The test code is often a mess and it deals with setting-up the aggregates as well the internals of an aggregate.
    • A NDepend-analyse revealed that our business-logic assembly will be a serious maintenance problem ('Zone of pain')

    Why is our business-logic in the zone of pain?

    We generally get closer to the 'zone of pain' (see link for details), when a high number of types in an assembly:

    1. are more concrete than abstract (see Abstractness in the formula)
    2. are used a lot by other assemblies (see Ca in the formula)
    3. do not depend on other types (see Ce in the formula)

    Since I can't see any value in implementing business-logic abstract (1.) and our business-logic already depends on a small number of types in other assemblies (2.) the only variable would be to reduce the number of types that are used by other assemblies (3.).

    How could we do it different?

    We could try to better encapsulate the internals of an aggregate avoiding the problems described above. That means, not allowing to access any internals by another type than the aggregate root. This increases the afferent coupling of the aggregate root, on the other hand eliminates the afferent coupling of the internals. The afferent coupling of the business logic assembly itself would then decrease.

    Therefore I transformed one of our aggregates from something like that

    // File Invoice.cs
    public class Invoice
    {
    public EntitySet<InvoicePosition> Positions { get; set; }
    }
    public class InvoicePosition
    {
    }

    to

    // File Invoice.cs
    public partial class Invoice
    {
    private EntitySet<Invoice.InvoicePosition> Positions { get; set; }
    }
    // File InvoicePosition.cs
    public partial class Invoice
    {
    private class InvoicePosition
    {
    }
    }
    After completing the refactoring and changing some logic in the presentation layer I got the following code metrics:

    1. The assembly containing the business logic moved a little bit away from the zone of pain (from D=0.577 to 0.575). This is still bad, however the refactoring had a positive effect.
    2. The type-rank for the refactored aggregate root increased from 10.7 to 12. Ca (Afferent coupling) increased from 62 to 66. That means, it got a more important part of the business logic.
    3. The CC (cyclomatic complexity) for the aggregate root increased from 38 to 77, while CC decreased generally for the presentation layer (e.g. in one case from 26 to 16). That means that we put complexity into the business logic. It's a good thing since the complexity generally is easier to test and therefore to maintain in the business logic. It was also possible to eliminate duplicated business logic found in the presentation layer.

    Testability and Unittests

    As the internals are completely hidden from outside now I test the refactored aggregate with state-based black-box tests. I think that's a reasonable way to do it since the test setup do not depend anymore on the implementation details of an aggregate. I'm not sure if this should be a general testing strategy for aggregates. I think there are more complex cases, where white-box tests would still be needed. The following pictures shows on the left, the test-code before the refactoring and on the right, the test-code after:

    Monday, July 14, 2008

    TDD, Design and Testability

    After being on holiday for a while and being pretty busy at work I want to talk about an upcoming issue at my current project. The issues are
    1. Presenter-Logic (Passiv View) is hard to test. It costs a lot of effort and it's difficult to be done right.
    2. There is code that is not worth being tested. We shouldn't test that code just to have the test coverage increased.

    As a TDD advocate I'm trying to find out what's about those two issues above:

    1.Passiv View is hard to test.
    Let see an example:
    public void EditDienstAbwesenheit()
    {
    var abwesenheit = View.SelectedDienstAbwesenheit; // get selected item from view
    if (abwesenheit != null) // if user as selected something
    {
    IModalDialog dialog;
    if(abwesenheit is SpitalAufenthalt) // decide what type of item is selected
    { // create a modal dialog for this item type
    var presenter = _viewBuilder.Create(_unitOfWork,typeof(ISpitalaufenthaltbearbeitenDialog));
    dialog = presenter.View;
    }
    else
    { // creat a modal dialog for the other item type
    var presenter = _viewBuilder.Create(_unitOfWork,typeof(IAbwesenheitbearbeitenDialog));
    dialog = presenter.View;
    }
    if (Shell.ShowInModalWorkspace(View, dialog) != DialogResult.OK) // display the modal dialog
    {
    _unitOfWork.Refresh(abwesenheit); // reload data the might have been modified
    }
    UpdateView(); // show the modification on the view
    }
    }


    We all know that the more dependencies and responsibilities a method has the more is it hard to test. This is like that because each dependency needs a mock or stub and each responsibility needs a test-assertion to verify it. The example above has following responsibilities:
    1. Get the users selection
    2. Decide about the next dialog to be shown
    3. Create the dialog
    4. Open the dialog in the shell
    5. Reload to rollback modifications of the dialog
    6. Present the changes to the users

    To accomplish that tasks we need:

    • a View to get the users input
    • a Model (called 'abwesenheit') that represent the users input
    • a ViewBuilder to create a Dialog
    • a Shell to display the Dialog
    • a UnitOfWork to rollback the modifications

    So what? Is it a test-problem or a design-problem? I strongly believe that code that can't be tested is poorly designed. The most obvious is that we violate the Single Responsiblity Principle (SRP). Could we split the responsibilities to find methods or even classes to separate the different concerns?

    Lets try:

    1. Get the users selection -> Lets keep it where it is
    2. Decide about the next dialog to be shown -> Lets put it into a separate method
    3. Create the dialog -> Lets delegate it to another object as creation and displaying a dialog is often used in an application (e.g Application controller)
    4. Open the dialog in the shell -> move it to the Application controller
    5. Reload to undo modifications -> That's wrong here. The dialogs are responsible not to modify data in case they are cancelled!
    6. Present the changes to the users -> Lets keep it where it is

    And after that minor refactoring:

    public void EditDienstAbwesenheit()
    {
    var abwesenheit = View.SelectedDienstAbwesenheit;
    if (abwesenheit != null)
    {
    CreateEditDialog(abwesenheit);
    UpdateView();
    }
    }
    public void CreateEditDialog(DienstAbwesenheit abwesenheit)
    {
    if(abwesenheit is SpitalAufenthalt)
    {
    _applicationController.ShowInModalWorkSpace(_unitOfWork,typeof(ISpitalAufenthaltBearbeitenDialog));
    }
    else
    {
    _applicationController.ShowInModalWorkSpace(_unitOfWork,typeof(IAbwesenheitBearbeitenDialog));
    }
    }

    So is that better? I think yes. In terms of Unit-Testing we reduced responsibility and dependency per unit (method) and we got a cleaner more expressive design.

    2. There is code that is not worth being tested
    Lets see following example:
    public class Person
    {
    public string FamilyName { get; set; }
    public string Name { get; set; }
    public string Age { get; set; }
    }
    public class Address
    {
    public string Code { get; set; }
    public string Country { get; set; }
    public string Street { get; set; }
    }
    /// <summary>
    /// Model
    /// </summary>
    public class PersonRow
    {
    public PersonRow(Address address, Person person)
    {
    _address = address;
    _person = person;
    }

    private readonly Person _person;
    private readonly Address _address;

    public string FamilyName
    {
    get { return _person.FamilyName; }
    }

    public string Name
    {
    get { return _person.Name; }
    }

    public string Age
    {
    get { return _person.Age; }
    }

    public string Code
    {
    get { return _address.Code; }
    }

    public string Country
    {
    get { return _address.Country; }
    }

    public string Street
    {
    get { return _address.Street; }
    }
    }


    Is it worth to test that each getter returns the right value? When is it worth to test a piece of code? I could think of following answers:

    1. It's worth if something can go wrong
    2. It's worth if I don't need to put a lot of effort to test it
    3. It's worth if it's likely to be refactored in future

    The problem with the example above is that little can go wrong (except the mapping) and that the effort is too high to test it. On the other hand as a TDD advocate I'm a little bit annoyed that this little stupid class should lower our test coverage. So what is the way out?

    • If this class were not necessary there shouldn't go anything wrong
    • If we could reduce the lines of code, less could go wrong and less needs to be tested

    So let's try to introduce a mapper to externalize the mapping:

    public class Person
    {
    public string FamilyName { get; set; }
    public string Name { get; set; }
    public string Age { get; set; }
    }
    public class Address
    {
    public string Code { get; set; }
    public string Country { get; set; }
    public string Street { get; set; }
    }
    /// <summary>
    /// Model
    /// </summary>
    public class PersonRow
    {
    public string PersonFamilyName { get; set; }
    public string PersonName { get; set; }
    public string PersonAge { get; set; }
    public string AddressCode { get; set; }
    public string AddressCountry { get; set; }
    public string AssressStreet { get; set; }
    }

    public void DoTheMapping()
    {
    var person = new Person();
    var address = new Address();
    var row = new PersonRow();
    PropertyMapper.To(row).From(person).From(address).Execute();
    }

    The idea is to eliminate all the boring mappings to a specialized class PropertyMapper. The PropertyMapper would map the values from person and address to the row combining Classname and Propertyname. Furthermore the method Execute() would throw an exception in case any property of the PersonRow was not initialized with a value. My observations are:
    1. The PropertyMapper is difficult to implement, but I think after 10 mappings i have already my return on investment
    2. Execute() is self-validating, therefore this code can also be run under test
    3. I split the responsibilities into two classes (PersonRow and PropertyMapper) and I got testability
    4. The example should be extended to support two way mappings
    5. Is that too much?

    Friday, March 21, 2008

    Some thoughts about C# 3.0

    We have been using the new C#3.0-features for two months and its quite interesting that our code looks in some parts like Jeremy describes in C#3.0 is like a moped. I wouldn't like to miss the new features and I won't stop using them but I'm thinking it's getting too much.

    It makes me feel that C# already had it's best time and there should be something new. Where is the new microsoft-ruby-language? C# was a better Java (I mean the language not the community) but where is now the better ruby?

    Sunday, January 13, 2008

    Behind the scences of LINQ

    I always knew that LINQ was founded on the idea of monads but even trying hard I couldn't understand how LINQ and monads are related to each other.

    But fortunately there is Wes Deyer and he describes in The Marvels of Monads what I always wanted to know. It's a pity that LINQ was always described and documented as query language rather than as a monad extensions. When I look to the msdn-documentation there is no hint that I could implement query operators on other types than IEnumerable and IQueryable.

    But that's exactly what Wes is doing. He sees a more general concept and he shows a way to implement the query operators on any type. In fact this is for me a huge step and leads us to a new level of abstraction where LINQ (as a list-monad) seems to be a specific case of a monad.

    My preferred way to use LINQ was the method syntax as I thought this is the more general case. But now, after knowing that the query syntax is something like a monad extension I start to like it.

    Saturday, December 29, 2007

    Playing around with NBehave

    As I said recently I'm still exploring tdd and its capabilities. The next evolutionary step could be bdd. Therefore I downloaded NBehave and I tried to write one story with it. Passive View is my preferred playground and thats how it goes with NBehave and TypeMock.NET:

    First the declaration of the View, the Presenter and a Service-Gateway that is used by the Presenter to retrieve some information.
    protected IView _view;
    protected Presenter _presenter;
    protected EventHandler _updateRequestedEvent;
    protected IServiceGateway _serviceGatway;

    The setup of the this objects is not showed here. It simples instantiates mocks for the View and the ServiceGateway and injects it into the Presenter using Constructor-Injection. The _updateRequestedEvent is a mocked event on the view and is also used later in the example.

    First I have to specify a story. This is done like that:

    [TestFixtureSetUp]
    public void FixtureSetup()
    {
    _getCurrentTimeStory = new Story("Getting the current service time");
    _getCurrentTimeStory.AsA("User").IWant("to update the displayed current time")
    .SoThat("I can see what time is on the serivce side");
    }

    And that's one scenario that came in my mind:

    [Test]
    public void HelloNTServiceIsNotAvailable()
    {
    using (RecordExpectations recorder = RecorderManager.StartRecording())
    {
    _view.ShowMessageBox("The Service is currently not available");
    recorder.CheckArguments();
    }
    _getCurrentTimeStory.WithScenario("Service is not available")
    .Given("the Service returns an exception", new Exception("Error"), e =>
    {
    using (RecordExpectations recorder = RecorderManager.StartRecording))
    {
    recorder.ExpectAndThrow(_serviceGatway.GetTime(), e);
    }
    })
    .When("I update the current time", () => _updateRequestedEvent(this, EventArgs.Empty))
    .Then("a message box should display", "The Service is currently not available", m =>
    {
    MockManager.Verify();
    })
    ;
    }

    Looks quite chaotic and a developer not used to NUnit,TypeMock and NBehave might not understand it at first glance. I also think the combination of NUnit,TypeMock and NBehave is a little bit strange as they do not integrate very well. Maybe there is another way how to combine those frameworks?

    But what I like is the overall structure that it gives to the tests and the readable test output that can be interpreted by a product owner. Here are now the scenario above and another scenario that I could present to him:

    Story: Getting the current service time

    Narrative:
    As a User
    I want to update the displayed current time
    So that I can see what time is on the serivce side

    Scenario 1: Service is not available
    Given the Service returns an exception: System.Exception: Error
    When I update the current time
    Then a message box should display: The Service is currently not available

    Scenario 2: Service is available
    Given the Service returns the time: 01.01.2000 12:24:23
    When I update the current time
    Then the label on the view should display: 12:24