Common thing in DB which supports ACID principle

I was just undergoing training on Neo4j db. It is basically a graph database which supports ACID principle.

I see it uses transaction logs similar to Microsoft SQL server. Whenever the server crashes it will roll back all transactions, and go back to a stable state.

First thing to support ACID is you need to keep track of transactions in some manner to roll back in case of failure. And that means there should be a transaction log supported. Point is if we hear ACID then there should be a transaction log.

 I will keep updating this blog to generalize design pattern around ACID systems. You are welcome to share your thoughts :)

 

Loosing Intent Of Code

Recently I was welcomed by my friend to comment on his code. He had two xml files which represent the object. His intention is to compare the two objects for its equality. The approach that was taken is to load two XML files and compare node by node. Working of the program was explained by the author. I could only follow the code with his comment. Intent of the program was lost under the rubble of programming constructs and framework objects. Let us see what we could have done to prevent that.

Problem

Let us take a similar example here. I am asked to compare two cars. Details of both cars are represented by XML file. I would write the program like as shown below.

 class Program
 {
   static void Main(string[] args)
   {
      var xmlDocument1 = ReadSourceXmlFile();
      var xmlDocument2 = ReadDestinationXmlFile();
      var equal = Compare(xmlDocument1, xmlDocument2);
   }

 private static bool Compare(XmlDocument xmlDocument1, XmlDocument xmlDocument2)
 {
   //..Compares both files
   return true;
 }

 private static XmlDocument ReadDestinationXmlFile()
 {
    //...Reads Xml file for Car1
    return new XmlDocument();
 }

 private static XmlDocument ReadSourceXmlFile()
 {
    //...Reads xml file for Car2
    return new XmlDocument();
 } 
 } 

If anyone is looking at the code does it make any sense? Even one who wrote will struggle to understand the intention behind this code. Ofcourse the comments could help to some extent. Problem here is code is talking interms of its own programming constructs and objects. It is not communicating anything about the real world problem it is solving. Now let us see how we can make it better.

Solution

 class Program
 {
   static void Main(string[] args)
   {
     var car1 = GetCarDetails("source file path");
     var car2 = GetCarDetails("destination file path");
     var equal = CompareCars(car1, car2);
    }

   private static bool CompareCars(Car car1, Car car2)
   {
     //..Compares both files
    return true;
   }

   private static Car GetCarDetails(string sourceFile)
   {
     //...Reads Xml file and build the car object
      return new Car();
    } 
 }

 public class Car
 {
    //..Car has its own properties
 }

Here the code is talking in terms of real world problems. It is taking two cars and comparing.

No one in real world will compare two files when they want to compare two cars :D

Advantages

I could instantly see some of the advantages with second approach. I am having a common set of functionality such as common function to build the car from xml file.

With intentions being clear we could even move ahead and break the responsibilities into separate set of classes. In this case building car object can be moved into CarObjectBuilder. Comparing cars can be the responsibility of CarComparer object. Both can change irrespective of each other reaping the benefit of single responsibility principle.

I know I could make the above program more jazzy by introducing the data provider for building the car object. Point is try to emulate the real world problems as is in the code. This paradigm shift in thinking interms of real world problem could help in building highly cohesive loosely coupled systems. Please let me know your thoughts.

Regards,

Nitin

Abstract Factory Vs Factory Method

Many times it seems like factory and factory method pattern are doing the same job. But the evil is hidden in details. Let me show you what I am talking about.

Car Factory Method Story

Consider a factory which is manufacturing cars. It builds different parts of the car such as Door, Steering, Body and Engine etc. Then it puts these parts on to the chassis.

CarFactory

Car factory method class looks like this

    public abstract class CarFactory
    {
        protected abstract string BuildSteering();

        protected abstract string BuildWheels();

        protected abstract string BuildDoors();

        public Car BuildCar()
        {
            var car = new Car();
            car.Steering = BuildSteering();
            Console.WriteLine("Steering is {0}",car.Steering);
            Console.WriteLine("Fit it with Bronze Bolts and Nuts on Chassis");
            car.Wheels = BuildWheels();
            Console.WriteLine("Wheels are {0}", car.Wheels);
            Console.WriteLine("Fit it with Bronze Bolts and Nuts on Chassis");
            car.Doors = BuildDoors();
            Console.WriteLine("Doors are {0}", car.Doors);
            Console.WriteLine("Fit it with Bronze Bolts and Nuts on Chassis");
            return car;
        }
    }    

Note that for all cars the parts are connected to chassis in the same way. All parts will be placed and fastened with bolts and nuts.

Company decided to manufacture sports car which has alloy wheels, fiber doors and power steering. All parts are manufactured and sent to assembly line for adding the parts. Now in assembly line same procedure of fitting these parts are followed (which is fastening with bronze bolts and nuts)

SportCar

So Sports car factory looks like

 public class SportsCarFactory : CarFactory
 {
   protected override string BuildDoors()
   {
     return "Fiber Doors";
   }

   protected override string BuildWheels()
   {
     return "Alloy wheels";
   }

   protected override string BuildSteering()
   {
    return "Power Steering";
   }
 }

codeTry the code here..

Competitor turned up :(

Company opened many similar factories with same assembly operations until one day competitor turned up.

Just like competitor, company decided to manufacture the sports cars and commercial cars differently. They feel instead of using bronze bolts and nuts for sports cars, they can use fiber bolts to give speed advantage. Yeah.. sure.. why not.. But what about Commercial cars? They are built with metal body, and they can not take fiber bolts. Errr… Problem :(

CommercialCar

Why problem? Because you can customize the parts, but can not customize the assembly operations.

 Into New Business with Abstract Factory :)

Company realized they need two different assembly operations for manufacturing cars.  But they are still building cars. Now car factory is just an abstraction. It doesn’t dictate any assembly operations.

 public abstract class CarFactory
 {
    protected abstract string BuildSteering();

    protected abstract string BuildWheels();

    protected abstract string BuildDoors();

    public abstract Car BuildCar();
 }

Now the Sports car factory and commercial car factory look like


 public class SportsCarFactory : CarFactory
    {
        protected override string BuildDoors()
        {
            return "Fiber Doors";
        }

        protected override string BuildWheels()
        {
            return "Alloy wheels";
        }

        protected override string BuildSteering()
        {
            return "Power Steering";
        }

        public override Car BuildCar()
        {
            var car = new Car();
            car.Steering = BuildSteering();
            Console.WriteLine("Steering is {0}", car.Steering);
            Console.WriteLine("Fit it with fiber Bolts and Nuts on Chassis");
            car.Wheels = BuildWheels();
            Console.WriteLine("Wheels are {0}", car.Wheels);
            Console.WriteLine("Fit it with fiber Bolts and Nuts on Chassis");
            car.Doors = BuildDoors();
            Console.WriteLine("Doors are {0}", car.Doors);
            Console.WriteLine("Fit it with fiber Bolts and Nuts on Chassis");
            return car;
        }
    }

    public class CommercialCarFactory : CarFactory
    {
        protected override string BuildDoors()
        {
            return "Metal Doors";
        }

        protected override string BuildWheels()
        {
            return "Alloy wheels";
        }

        protected override string BuildSteering()
        {
            return "Power Steering";
        }

        public override Car BuildCar()
        {
            var car = new Car();
            car.Steering = BuildSteering();
            Console.WriteLine("Steering is {0}", car.Steering);
            Console.WriteLine("Fit it with bronze Bolts and Nuts on Chassis");
            car.Wheels = BuildWheels();
            Console.WriteLine("Wheels are {0}", car.Wheels);
            Console.WriteLine("Fit it with bronze Bolts and Nuts on Chassis");
            car.Doors = BuildDoors();
            Console.WriteLine("Doors are {0}", car.Doors);
            Console.WriteLine("Fit it with bronze Bolts and Nuts on Chassis");
            return car;
        }
    }

codeTry the code here

We turned away from factory method to abstract factory.

Let me know what you guys think.. :)

Blame the Technology..

This is an instance from days when I was spending time in office late night root causing hidden bugs. Our program was running all fine for a month, then used to crash with out of memory exceptions. First response from me is “this .NET framework has bug”. We started root causing it with the help of available memory profilers. Slowly the ugly truth revealed. Its not the framework, but the piece of code which was not in right place was creating lots of weak handles in memory. Finally the memory crash. Fixing the code kept our system for months until any one takes the server for maintenance.

Second instance is when the MSMQ message sent appeared late at destination. Again my first opinion is MSMQ has BUG. We did talk to MS people sending them memory dumps over email. Meanwhile I was reading MSDN documents about MSMQ. After reading forums and documents I learnt that MSMQ is built around TCP/IP. When there are no messages the TCP/IP connection is broken, and re-established when a message arrives. Establishing this TCP/IP connection was taking sometime due to Active Directory authentication. After understanding the underlying design of MSMQ, I found a registry entry for this connection duration. This registry entry was modified to hold the connection for a day. And result was awsome.. Never again we saw the problem. Again I am proved wrong judging the technology and understanding the design.

Now this is an instance from current days. We had a .NET program running on top of communication framework. The framework used MSMQ as transport. The transport was changed to SQL server. Suddenly the program which was performing very well started slowing down. Again I blamed the technology. There was a challenge lying in front of me to fix it. Discussing with team and insight into the business logic showed the way. The program was duplicating the processing n number of times though the response for each input was same. Cutting down that extra processing brought down the latency by 80%.

 In all these situations my first response was to blame the technology. But in the end I am able to get the program healthy. Moral of the story is focus on problem, not on the technology. We can fix either the technology or the design to get the things working.

 Let me know what you guys think :)

Single Responsibility Principle – I Hate It

I am fond of Single Responsibility Principle.. But as the day passed I developed distaste for this principle. Why? The application I was working on grew with huge number of classes and interfaces. Code seemed like a spaghetti code. I started thinking that this principle instead of solving problems adding more to it. None the less the application became loosely coupled but grew into something that is difficult to understand. I would say the code became less cohesive. entangled wires

But I began realizing that the principle is not wrong, but the way I perceived it was wrong. When single responsibility says there should be one responsibility and one reason for change, the questions are

1. What is going to change?

2. What is the responsibility boundary? Is it class, collection of classes, module, collection of module?

If you take the WCF Service which serves product, is it not that the Product service following single responsibility?. When it changes it changes more for the reason that something related to product changed.

Or lets take an example of Employee detail. In order to display employee detail we go through employee UI, employee business layer, data access layer. Each layer is following SRP.

 If you take the employee detail class, you could have methods like GetJoiningDate, GetRelievingDate, GetPromotedDate, GetSalary, GetIncomeTaxOnSalary. All related to employee and I can argue that the only reason it could change is when I want to modify the employee related rules. That is perfectly normal.

And instead what I did is I over imagined that Dates and Salary are not same thing of an employee. Why not break it into EmployeeSalary and EmployeeChronology? We got 2 classes now. And when I am thinking in-terms of EmployeeSalary I feel that IncomeTax could be calculated annually and monthly. OOpss.. we need another class EmployeeIncomeTax.  Now we got 3 classes. Now employee could be rewarded with bonus. Argh.. Yes we need another class. EmployeeBonus. We made 4 classes. Employee could leave and we need to calculate his settlement and dues.. Ah.. Its a good idea to break it into another class EmployeeSettlement. We got 5 classes. See.. this is what happens when single responsibility boundary is not fixed :)

But notice that all these classes in reality are representing behaviors. Does this behavior change more often? Do you think you need to break classes this way where behavior wont change in another 100 years? Design principle says what changes should be kept apart from what doesn’t change.

If its not going to change then I am happy with Employee detail class. If the way the bonus is calculated more often changes , you got to change the logic more often, then move it apart.. How often bonus rules change is the call of the business.

Now easier it is to move the parts apart as long as you maintain unit tests. Otherwise you are on your own when playing something like this.

Thanks for reading.. Let me know your thoughts.

 

Writing Custom Framework – Good Or Bad

As per wiki framework is “an abstraction in which software providing generic functionality can be selectively changed by additional user-written code, thus providing application-specific software”  Interesting here is to see the word abstraction. What is abstracted by the software? 

Windows Communication Foundation (WCF) abstracts communication mechanism. Developer can use this framework to build applications which can talk to each other. WCF abstracted various communication mechanism such as MSMQ, HTTP, TCP. Also the framework addressed the concerns such as security, serialization that revolve around the communication requirements.

Similarly there are various frameworks for building UI such as MVC, ASP.NET, WPF so on. On top of these framework people try to write their own custom framework. Ofcourse the custom framework written on top of these various framework do fit the domain, but did anyone  consider the downside of it?

One of My experience with custom framework is awful. The framework offered nice features such as caching, hosting service and what not. I had installed windows azure bus on my machine. And the custom framework I was using started breaking. Having no clue and no one to go to, I had to debug into the framework code. I did find a place where this heavy weight custom framework is scanning all communication channels. Framework picked azure bus as its communication mode, and was missing configuration for it. I am sure the guys who wrote framework missed this scenario which could happen, and did not add configuration for it.  

What is the result of this mishap? The framework which is intended to reduce development time added hurdles to it. I spent first half of day contacting the framework developers, and then second half of day debugging the framework myself. 

The funny part is framework developers could not root cause the problem because there was no windows azure bus installed on their machine. More embarrassing is to say “it doesn’t work on my machine, but works everywhere else”

Like the framework definition goes the framework should solve the problem for only one abstraction (single responsibility). Does wiki say abstractionS? Should the framework choose the communication channel for the user, or user should choose the channel :)? 

 

Testable Objects

I have seen companies where unit testing is a theory than practice. Some think that creating a unit testing project in Visual Studio is unit testing.

What is unit testing then? Like a building constructed brick by brick, software is built with classes. Similar to faulty bricks can cause damage to the structure of the building, a badly written class can harm the application code.

I started my career writing bad code, and improved my coding style with constant learning under great Teachers. I know what is a bad code because I have written lot of it :D

So how do we write testable code?

Lets take a simple example of layered architecture which is very much prevalent in the industry. A three layer architecture will have Data Access Layer (DAL), Business Layer (BL) and Presentation Layer (PL).
So sample code looks like as shown below.


class Program
    {
        static void Main(string[] args)
        {
            PresentationLayer presentationLayer = new PresentationLayer();
            presentationLayer.Call();
        }
    }

    class PresentationLayer
    {
        BusinessLayer _businessLayer = new BusinessLayer();

        public void Call()
        {
            Console.WriteLine("Presentation Layer Called");
            _businessLayer.Call();
        }
    }

    class BusinessLayer
    {
        DataAccessLayer _dataAccessLayer = new DataAccessLayer();
        public void Call()
        {
            Console.WriteLine("Business Later Called");
            _dataAccessLayer.Call();
        }
    }

    class DataAccessLayer
    {
        public void Call()
        {
            Console.WriteLine("Data Access Layer Called");
        }
    }

Presentation layer makes a call to business logic layer, which in turn calls data access layer. Now unit testing means we should be able to test single functionality of one class. Can we test the call in business layer without having reference to data access layer? To make matter worse data access layer will mostly be written tightly coupled with database. So you can not test business layer until you have the  data access layer and database.

Still this is a not a convincing answer for having unit test? Then answer to question “how do you test all business scenarios around the BL?” could be more reasonable :)

Without unit test, business scenarios will be lost under the pile of code. And no one in the team can be sure that they did not break the component they touched.

To make the component testable one of the technique is to use Dependency Inversion Principle. With dependency inversion principle, we are going to compose the class as shown below.


class Program
    {
        static void Main(string[] args)
        {
            IDataAccessLayer dataAccessLayer = new DataAccessLayer();

            IBusinessLayer businessLayer = new BusinessLayer(dataAccessLayer);

            //We are composing the object
            PresentationLayer presentationLayer = new PresentationLayer(businessLayer);

            presentationLayer.Call();
        }
    }

    class PresentationLayer
    {
        IBusinessLayer _businessLayer ;

        public PresentationLayer(IBusinessLayer businessLayer)
        {
            _businessLayer = businessLayer;
        }

        public void Call()
        {
            Console.WriteLine("Presentation Layer Called");
            _businessLayer.Call();
        }
    }

    interface IBusinessLayer
    {
        void Call();
    }

    class BusinessLayer : IBusinessLayer
    {
        IDataAccessLayer _dataAccessLayer;

        public BusinessLayer(IDataAccessLayer dataAccessLayer)
        {
            _dataAccessLayer = dataAccessLayer;
        }

        public void Call()
        {
            Console.WriteLine("Business Layer Called");
            _dataAccessLayer.Call();
        }
    }

    interface IDataAccessLayer
    {
        void Call();
    }

    class DataAccessLayer : IDataAccessLayer
    {
        public void Call()
        {
            Console.WriteLine("Data Access Layer Called");
        }
    }

Our sample unit test around business layer with a stub data access layer is as shown below.


[TestClass]
    public class UnitTest1
    {
        public class DataAccessStub : IDataAccessLayer
        {
            public void Call()
            {
                Console.WriteLine("Data access stub called");
            }
        }

        [TestMethod]
        public void TestCall()
        {
            IDataAccessLayer dataAccessLayer = new DataAccessStub();

            IBusinessLayer businessLayer = new BusinessLayer(dataAccessLayer);

            businessLayer.Call();
        }
    }

Instead of using database dependent data access layer, we are using DataAccessLayerStub. We can now test all business scenarios around the business layer with the help dependency inversion principle.

I will explain more about stub in my next post. So keep watching :)